Making it harder to do easy things, but easier to do harder things with the Pytorch Dataloader
About • Installation • Usage • Development • License
BetterLoader is a hyper-customizable extension of the default PyTorch dataloader class, that allows for custom transformations pre-load and image subset definitions. Use the power of custom index files to maintain only a single copy of a dataset with a fixed, flat file structure, and allow BetterLoader to do all the heavy lifting.
pip install betterloaderBetterLoader allows you to dynamically assign images to labels, load subsets of images conditionally, perform custom pretransforms before loading an image, and much more.
A few points worth noting are that:
- BetterLoader does not expect a nested folder structure. In its current iteration, files are expected to all be present in the root directory.
- Every instance of BetterLoader requires an index file to function. Sample index files may be found here.
from betterloader import BetterLoader
index_json = './examples/sample_index.json'
basepath = "./examples/sample_dataset/"
batch_size = 2
loader = BetterLoader(basepath=basepath, index_json_path=index_json)
dataloaders, sizes = loader.fetch_segmented_dataloaders(batch_size=batch_size, transform=None)
print("Dataloader sizes: {}".format(str(sizes)))For more information and more detailed examples, please check out the BetterLoader docs!
We use Makefile to make our lives a little easier :)
make installmake samplemake testDistributed under the MIT license. See LICENSE for more information.