-
Notifications
You must be signed in to change notification settings - Fork 103
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reduce package distribution size #1769
Conversation
Test "pass". All of the failures are in NIRSPEC, presumably because of recent changes to the raw data available. I'll re-run after the NIRSPEC branch is merged.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is impressive @kbwestfall - job very well done! I only have a few minor points, but the new approach is a lot better than before. I also have one general question... If a user updates their version of PypeIt, will this require them to re-download all data files when needed (or, only if those data files have been updated)? It just wasn't clear to me how any updated (i.e. changed from one release to the next release) data files are being managed. Thanks again!
I have no major issues with this PR, so I'm approving.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great. Thanks for addressing the Windows path questions.
Merges develop into release (1.16.0 tag prep)
There is a test failure, but it's because of a recent change to the files in the relevant Keck/HIRES dataset. The vet-test failure is related. So I think this one is ready to go.
|
The primary goal of this PR is to reduce the size of the package distribution installed using pip. To do so, I
For item 1, I added the
pypeitdata.py
andcache.py
files, which are refactored pieces of what wasdata/utils.py
, and then I moved some of theload_*
functions fromdata/utils.py
intoio.py
. With this refactor, I've removed thepypeit.data
module and replaced it with thepypeit.dataPaths
instance; see the changes to the main__init__.py
file. If this is accepted, I'd like to make thedata/
directory exclusively for data files; i.e., it should not have any code in it.After some testing, this reduces the size of the tar file from ~145MB for version 1.15.0 to about 12MB. The big ticket items were the test data files, some of the standards, and doc figures directory.
There's no hurry on this one, but I wanted to issue this so that everyone can start to comment.