Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Finalize top-level API #46

Merged
merged 6 commits into from
Jul 14, 2023

Conversation

j-bennet
Copy link
Collaborator

@j-bennet j-bennet commented Jul 12, 2023

  • Finalize API for writing to deltalake
  • Rename read_delta_table -> read_deltalake
  • Remove vacuum and history; user can use deltalake.DeltaTable for that
  • Add documentation for to_deltalake to README

Closes #42, #17, #16.

@j-bennet j-bennet marked this pull request as draft July 12, 2023 16:57
@j-bennet j-bennet force-pushed the j-bennet/finalize-write-api branch from 2aa861e to 29dbee8 Compare July 12, 2023 16:57
@j-bennet j-bennet changed the title Finalize write api Finalize top-level API Jul 12, 2023
@j-bennet j-bennet marked this pull request as ready for review July 12, 2023 17:46
@codecov-commenter
Copy link

codecov-commenter commented Jul 12, 2023

Codecov Report

Merging #46 (6a0d744) into main (5fe7c37) will decrease coverage by 2.43%.
The diff coverage is 100.00%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@            Coverage Diff             @@
##             main      #46      +/-   ##
==========================================
- Coverage   76.63%   74.21%   -2.43%     
==========================================
  Files           6        6              
  Lines         351      318      -33     
==========================================
- Hits          269      236      -33     
  Misses         82       82              
Impacted Files Coverage Δ
dask_deltatable/__init__.py 100.00% <100.00%> (ø)
dask_deltatable/core.py 91.25% <100.00%> (-2.67%) ⬇️
dask_deltatable/write.py 65.51% <100.00%> (+1.23%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@j-bennet
Copy link
Collaborator Author

Windows tests fail uploading coverage to codecov.

@j-bennet j-bennet force-pushed the j-bennet/finalize-write-api branch from 569e019 to 41170bb Compare July 12, 2023 20:54
dask_deltatable/core.py Outdated Show resolved Hide resolved
@@ -21,9 +21,6 @@ pip install dask-deltatable
- Parquet filters
- row filter
- partition filter
5. Query Delta commit info and history
6. API to ``vacuum`` the old / unused parquet files
7. Load different versions of data by timestamp or version.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still true, isn't it?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@j-bennet j-bennet requested a review from fjetter July 13, 2023 17:51
@j-bennet j-bennet force-pushed the j-bennet/finalize-write-api branch from 0e5cd50 to 6a0d744 Compare July 13, 2023 22:33
@fjetter fjetter merged commit a962ac3 into dask-contrib:main Jul 14, 2023
10 checks passed
@j-bennet j-bennet deleted the j-bennet/finalize-write-api branch July 14, 2023 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Finalize API for writing Delta Tables
3 participants