Skip to content

Code library that uses S3's API to provide an efficient random-access (seekable) Stream implementation for use in code where efficient network I/O is paramount.

License

Notifications You must be signed in to change notification settings

mlhpdx/seekable-s3-stream

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Seekable S3 Stream

Some S3 objects are big enough that working on them in memory isn't desirable, or even possible. This library performs selective, efficient data transfer from S3 that is orders of magnitude faster and more efficient than naiively using MemoryStream while maintaining compatibility with libraries and packages that work with a Stream interface. Examples for reading ISO, Zip, JPG and Parquet files are included in the repo.

For the full explaination, check out the article on Medium.

About

Code library that uses S3's API to provide an efficient random-access (seekable) Stream implementation for use in code where efficient network I/O is paramount.

Topics

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Languages