Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

repo: add helper method for doing data index lookup in subrepos #9708

Merged
merged 3 commits into from
Jul 12, 2023

Conversation

pmrowla
Copy link
Contributor

@pmrowla pmrowla commented Jul 6, 2023

Thank you for the contribution - we'll try to review it as soon as possible. πŸ™

Closes #9698

@pmrowla pmrowla self-assigned this Jul 6, 2023
@pmrowla pmrowla changed the title [WIP] repo: open subrepos directly with Repo.open(url, path=...) repo: add helper method for doing data index lookup in subrepos Jul 11, 2023
@pmrowla
Copy link
Contributor Author

pmrowla commented Jul 11, 2023

After looking at this some more, we don't actually want to open subrepo directly in dvc.api use cases, the caller will always be starting with the path relative to the git/scm root URL, and not relative to the subrepo nested root.

What we really want is just a helper method that takes the original path (either relative to git/scm root URL or just a standard localfs path) and then returns the correct data index + entry for that path (taking into account subrepos when necessary).

# pylint: disable-next=protected-access
key = fs._get_key_from_relative(fs_path)
# pylint: disable-next=protected-access
subrepo, _, key = fs._get_subrepo_info(key)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This subrepo lookup still feels like it belongs in Repo somewhere and not dvcfs, but at least this way the internal lookups are kept within the Repo level helper method and we don't need to touch the dvcfs internals in dvc.api methods

@codecov
Copy link

codecov bot commented Jul 11, 2023

Codecov Report

Patch coverage: 82.14% and project coverage change: -0.27 ⚠️

Comparison is base (1b2291c) 90.44% compared to head (eba72b9) 90.18%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9708      +/-   ##
==========================================
- Coverage   90.44%   90.18%   -0.27%     
==========================================
  Files         480      480              
  Lines       36481    36468      -13     
  Branches     5249     5243       -6     
==========================================
- Hits        32996    32888     -108     
- Misses       2892     2962      +70     
- Partials      593      618      +25     
Impacted Files Coverage Ξ”
dvc/repo/open_repo.py 85.10% <77.77%> (+2.60%) ⬆️
dvc/repo/__init__.py 94.41% <78.57%> (-0.63%) ⬇️
dvc/api/data.py 78.87% <100.00%> (-4.47%) ⬇️
dvc/fs/dvc.py 95.18% <100.00%> (ΓΈ)

... and 25 files with indirect coverage changes

β˜” View full report in Codecov by Sentry.
πŸ“’ Do you have feedback about the report comment? Let us know in this issue.

@pmrowla pmrowla marked this pull request as ready for review July 11, 2023 08:14
@pmrowla pmrowla merged commit a5dda1b into iterative:main Jul 12, 2023
18 of 20 checks passed
@pmrowla pmrowla deleted the open-repo-subdir branch July 12, 2023 05:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

repo: properly support monorepo/subrepos in open_repo/erepo
1 participant