Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove vresutil #1220

Merged
merged 10 commits into from
Dec 9, 2024
Merged

Remove vresutil #1220

merged 10 commits into from
Dec 9, 2024

Conversation

ekatef
Copy link
Member

@ekatef ekatef commented Dec 2, 2024

Closes #1211 replacing vresutil dependencies.

To give some context, vresutil dependencies have been dropped with #803, but re-appeared due to flaws when merging the sector-coupled version.

Changes proposed in this Pull Request

  • reimplemented annuity calculations
  • replaced vresutil toolset with shapely ones when calculating cell areas when building the population layouts.

Checklist

  • I consent to the release of this PR's code under the AGPLv3 license and non-code contributions under CC0-1.0 and CC-BY-4.0.
  • I tested my contribution locally and it seems to work fine.
  • Code and workflow changes are sufficiently documented.
  • Newly introduced dependencies are added to envs/environment.yaml and doc/requirements.txt.
  • Changes in configuration options are added in all of config.default.yaml and config.tutorial.yaml.
  • Add a test config or line additions to test/ (note tests are changing the config.tutorial.yaml)
  • Changes in configuration options are also documented in doc/configtables/*.csv and line references are adjusted in doc/configuration.rst and doc/tutorial.rst.
  • A note for the release notes doc/release_notes.rst is amended in the format of previous release notes, including reference to the requested PR.

@ekatef ekatef marked this pull request as ready for review December 2, 2024 17:13
@ekatef
Copy link
Member Author

ekatef commented Dec 2, 2024

As a technical comment, the revised implementation for calculating cell_areas basically updates the pyproj-based vresutil methods for pyproj >= 2.1.0. There is a nice example on the differences of two approaches.

Testing has resulted in non-zero differences for the old and new approaches, though the differences are less than 0.5% which is probably fine. To facilitate review, have added a comment block with the old implementation including vresutil part.

@ekatef
Copy link
Member Author

ekatef commented Dec 6, 2024

@davide-f it would be great to have your review on that as it relates to the fixing the environment issues. Would be great to resolve it before the release

Copy link
Member

@davide-f davide-f left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR looks great and CI passes.
It deserves adding a major release probably as the env is changed.
Fine for me then :) great catch!

Comment on lines 77 to 93
# ------------------------------------------------------------------------
# old implementation
# ------------------------------------------------------------------------
# from functools import partial
# from shapely.ops import transform, cascaded_union

# def area(geom):
# return reproject(geom).area

# def reproject(geom, fr=pyproj.Proj(proj='longlat'), to=pyproj.Proj(proj='aea', lat_1=33., lat_2=72.)):
# reproject_pts = partial(pyproj.transform, fr, to)
# return transform(reproject_pts, geom)

# in km^2
with mp.Pool(processes=snakemake.threads) as pool:
cell_areas = pd.Series(pool.map(vshapes.area, grid_cells)) / 1e6
# with mp.Pool(processes=snakemake.threads) as pool:
# cell_areas = pd.Series(pool.map(vshapes.area, grid_cells)) / 1e6
# ------------------------------------------------------------------------
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we drop this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think yes 😄 Done!

@davide-f davide-f mentioned this pull request Dec 9, 2024
8 tasks
@ekatef
Copy link
Member Author

ekatef commented Dec 9, 2024

The PR looks great and CI passes. It deserves adding a major release probably as the env is changed. Fine for me then :) great catch!

Thanks a lot @davide-f! The PR just adds a bit to the work you have done to accommodate the fixed environments 🙂 Have added a release note if you think the PR is worth it 😉

Agree, that is time for a major release. If CI will be still happy, I'll merge this PR and start preparations for that, as we discussed. Many thanks for the coordination and all the fantastic work you are doing.

@davide-f davide-f merged commit 5d02b29 into pypsa-meets-earth:main Dec 9, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Drop vresutil dependency
2 participants