You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Right now, we have a configure_scheduling that allows us to enforce parallelism such that replications is disallowed for values with memory usage > 0. This means that only things like small values or the result of some computation can be replicated. But if you have an ML model, for example, that has significant memory usage and needs to be replicated, then we currently don't have a good way of testing that with parallelism enforced.
Note: this is just with regards to testing where we want to parallelize as if we had larger data - this works fine if you actually have large data.
The text was updated successfully, but these errors were encountered:
Right now, we have a
configure_scheduling
that allows us to enforce parallelism such that replications is disallowed for values with memory usage > 0. This means that only things like small values or the result of some computation can be replicated. But if you have an ML model, for example, that has significant memory usage and needs to be replicated, then we currently don't have a good way of testing that with parallelism enforced.Note: this is just with regards to testing where we want to parallelize as if we had larger data - this works fine if you actually have large data.
The text was updated successfully, but these errors were encountered: