You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Improve the way we copy a Memmap from one place to another, if we want and if we don't want to change the filename
simplify the logic of MemmapTensor so that extracting a tensor is less confusing (i.e. check as_tensor, _tensor, _load_item and how they interact).
the device attribute is confusing, as the data is never really on another device but gets loaded there if needed. The usage is that one can put the device attr as a destination, then move the Memmap from process to process and when accessing the data it'll be on the right device. However, this may give the false illusion that the content of the tensor is stored on the device...
Describe the bug
We need to do some woodwork on MemmapTensor:
MemmapTensor(memmap_tensor[idx])
returns the wrong view of theMemmapTensor
[Feature] apply across multiple TDs #193MemmapTensor
creation args, including: creating aMemmap
on a specific location ([Feature] More arguments for MemmapTensor construction #189)Memmap
from one place to another, if we want and if we don't want to change the filenameMemmapTensor
so that extracting a tensor is less confusing (i.e. checkas_tensor
,_tensor
,_load_item
and how they interact).device
attribute is confusing, as the data is never really on another device but gets loaded there if needed. The usage is that one can put the device attr as a destination, then move the Memmap from process to process and when accessing the data it'll be on the right device. However, this may give the false illusion that the content of the tensor is stored on the device...cc @tcbegley
The text was updated successfully, but these errors were encountered: