Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try using libmamba solver for conda packaging #562

Merged
merged 3 commits into from
Oct 16, 2023
Merged

Conversation

psavery
Copy link
Collaborator

@psavery psavery commented Oct 10, 2023

It is officially supported by conda, and it is significantly faster.

Before this commit, Windows packaging usually takes 1 hour and 15 minutes. Mac packaging usually takes about 45 minutes. Linux usually takes about 35 minutes.

We'll see if there are any noticeable differences.

Fixes: #556

It is [officially supported by conda](https://www.anaconda.com/blog/conda-is-fast-now),
and it is significantly faster.

Before this commit, Windows packaging usually takes 1 hour and 15 minutes. Mac packaging
usually takes about 45 minutes. Linux usually takes about 35 minutes.

We'll see if there are any noticeable differences.

Signed-off-by: Patrick Avery <patrick.avery@kitware.com>
@psavery psavery force-pushed the packaging-use-libmamba branch 2 times, most recently from 2c4e82a to e495efd Compare October 10, 2023 21:48
@psavery
Copy link
Collaborator Author

psavery commented Oct 11, 2023

I got libmamba working for the Linux packaging, and it reduced the time by about half to about 17 minutes! That's a great speedup.

However, I haven't figured out yet how to get it working for Windows and Mac. I am setting up the solver to be libmamba, but conda build is apparently still not using the libmamba solver.

@psavery psavery force-pushed the packaging-use-libmamba branch 3 times, most recently from 61ee3c3 to 1aae1a0 Compare October 11, 2023 21:30
Signed-off-by: Patrick Avery <patrick.avery@kitware.com>
@psavery psavery force-pushed the packaging-use-libmamba branch 2 times, most recently from 8e1e6e2 to 48ad592 Compare October 11, 2023 23:05
It worked in the last commit, let's see if we can simplify it.

Signed-off-by: Patrick Avery <patrick.avery@kitware.com>
@psavery
Copy link
Collaborator Author

psavery commented Oct 13, 2023

Well, for Mac and Windows, I had to install libmamba twice, and set it in the environment twice. I'm not sure why, but I've spent an hour or two trying to figure it out and was unable to figure it out. It has something to do with conda build's internal processes, which I do not know much about. I don't have time to continue exploring this, so we will leave it as is.

Anyways, it does make the packaging much faster for all operating systems! The packaging is nearly twice as fast for every OS. Windows took 40 minutes instead of 75 minutes. Mac took 18 minutes instead of 45 minutes. And Linux took 15 minutes instead of 35 minutes. This PR is ready to go! We should do something similar for HEXRDGUI.

@psavery psavery requested a review from bnmajor October 13, 2023 17:42
psavery added a commit to HEXRD/hexrdgui that referenced this pull request Oct 13, 2023
This will potentially run significantly faster, as also demonstrated
in HEXRD/hexrd#562.

Signed-off-by: Patrick Avery <patrick.avery@kitware.com>
Copy link
Collaborator

@bnmajor bnmajor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic!! LGTM!

@psavery psavery merged commit 11fae15 into master Oct 16, 2023
6 checks passed
@psavery psavery deleted the packaging-use-libmamba branch October 16, 2023 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Conda installation often takes too long
2 participants