Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: implement file blockstore #660

Merged
merged 20 commits into from
Jan 9, 2025
Merged

feat: implement file blockstore #660

merged 20 commits into from
Jan 9, 2025

Conversation

cernicc
Copy link
Member

@cernicc cernicc commented Dec 30, 2024

Description

This is an implementation of FileBlockstorage used to persist car blocks directly to a file on disk.

Checklist

  • Make sure that you described what this change does.
  • Have you tested this solution?
  • Did you document new (or modified) APIs?

@cernicc cernicc added this to the Phase 3 milestone Dec 30, 2024
@cernicc cernicc self-assigned this Dec 30, 2024
th7nder
th7nder requested changes Dec 30, 2024
View reviewed changes
mater/lib/src/stores/file.rs Outdated Show resolved Hide resolved
mater/lib/src/stores/file.rs Outdated Show resolved Hide resolved
mater/lib/src/stores/file.rs Outdated Show resolved Hide resolved
mater/lib/src/stores/file.rs Outdated Show resolved Hide resolved
mater/lib/src/stores/file.rs Show resolved Hide resolved
mater/lib/src/v2/mod.rs Show resolved Hide resolved
mater/lib/src/stores/file.rs Show resolved Hide resolved
@cernicc cernicc requested a review from th7nder December 31, 2024 10:26
th7nder
th7nder previously approved these changes Jan 2, 2025
View reviewed changes
mater/lib/src/stores/file.rs Outdated Show resolved Hide resolved
jmg-duarte
jmg-duarte reviewed Jan 2, 2025
View reviewed changes
Copy link
Contributor

@jmg-duarte jmg-duarte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still some stuff to be cleared

mater/lib/src/stores/file.rs Outdated Show resolved Hide resolved
mater/lib/src/stores/file.rs Show resolved Hide resolved
mater/lib/src/stores/file.rs Show resolved Hide resolved
mater/lib/src/stores/file.rs Outdated Show resolved Hide resolved
mater/lib/src/stores/file.rs Outdated Show resolved Hide resolved
mater/lib/src/stores/file.rs Outdated Show resolved Hide resolved
mater/lib/src/stores/file.rs Outdated Show resolved Hide resolved
mater/lib/src/stores/file.rs Outdated Show resolved Hide resolved
mater/lib/src/stores/file.rs Outdated Show resolved Hide resolved
mater/lib/src/stores/file.rs Outdated Show resolved Hide resolved
@cernicc cernicc requested review from jmg-duarte and th7nder January 3, 2025 09:25
jmg-duarte
jmg-duarte reviewed Jan 3, 2025
View reviewed changes
Copy link
Contributor

@jmg-duarte jmg-duarte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more question: the CAR file should be able to support multiple files inside it, did you test that?

mater/lib/src/stores/file.rs Outdated Show resolved Hide resolved
mater/lib/src/stores/file.rs Outdated Show resolved Hide resolved
th7nder
th7nder previously approved these changes Jan 3, 2025
View reviewed changes
@cernicc
Copy link
Member Author

cernicc commented Jan 3, 2025 •
edited
Loading

One more question: the CAR file should be able to support multiple files inside it, did you test that?

It should work because the store is not doing any additional processing. It stores the received blocks to the underlying car file.

@cernicc cernicc marked this pull request as draft January 6, 2025 08:44
@cernicc cernicc requested a review from th7nder January 6, 2025 13:41
@jmg-duarte jmg-duarte added the ready for review Review is needed label Jan 6, 2025
jmg-duarte
jmg-duarte previously approved these changes Jan 6, 2025
View reviewed changes
mater/lib/src/multicodec.rs Outdated Show resolved Hide resolved
mater/lib/src/stores/file.rs Outdated Show resolved Hide resolved
mater/lib/src/stores/file.rs Show resolved Hide resolved
mater/lib/src/stores/file.rs Show resolved Hide resolved
mater/lib/src/multicodec.rs Outdated Show resolved Hide resolved
mater/lib/src/stores/file.rs Outdated Show resolved Hide resolved
mater/lib/src/stores/file.rs Outdated Show resolved Hide resolved
@cernicc cernicc added ready for review Review is needed and removed ready for review Review is needed labels Jan 6, 2025
@cernicc cernicc requested review from jmg-duarte and aidan46 and removed request for aidan46 January 7, 2025 08:34
jmg-duarte
jmg-duarte previously approved these changes Jan 9, 2025
View reviewed changes
th7nder
th7nder previously approved these changes Jan 9, 2025
View reviewed changes
@cernicc cernicc enabled auto-merge (squash) January 9, 2025 12:31
@cernicc cernicc dismissed stale reviews from th7nder and jmg-duarte via 2bb1f78 January 9, 2025 12:54
@cernicc cernicc requested review from jmg-duarte and th7nder January 9, 2025 15:00
@cernicc cernicc added ready for review Review is needed and removed ready for review Review is needed labels Jan 9, 2025
@cernicc cernicc merged commit 31533cd into develop Jan 9, 2025
5 checks passed
@cernicc cernicc deleted the feat/file_blockstore branch January 9, 2025 16:38
@cernicc cernicc mentioned this pull request Jan 16, 2025
Closed
@cernicc cernicc linked an issue Jan 17, 2025 that may be closed by this pull request
Implement mater file blockstore #653
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jmg-duarte jmg-duarte jmg-duarte approved these changes

@th7nder th7nder th7nder approved these changes

@pete-eiger pete-eiger Awaiting requested review from pete-eiger

@aidan46 aidan46 Awaiting requested review from aidan46

Assignees

@cernicc cernicc

Labels
ready for review Review is needed
Projects
None yet
Milestone
Phase 3
Development

Successfully merging this pull request may close these issues.

Implement mater file blockstore
3 participants
@cernicc @th7nder @jmg-duarte