Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document node's store structure #1603

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

Mackenzie-OO7
Copy link
Contributor

@Mackenzie-OO7 Mackenzie-OO7 commented Jun 10, 2024

Overview

closes #1341

Summary by CodeRabbit

  • New Features
    • Added "Datastore structure" link to the sidebar for improved navigation in the documentation.
    • Introduced a detailed overview of the Celestia node's datastore structure in the how-to guide, enhancing user understanding of data management.

Copy link
Contributor

coderabbitai bot commented Jun 10, 2024

Walkthrough

The recent updates enhance the documentation for the Celestia node by adding a sidebar link titled "Datastore structure" for improved navigation. This entry links to the path "how-to-guides/celestia-node-store-structure," providing users with a reference to the datastore structure related to the Celestia node.

Changes

Files Change Summary
.vitepress/config.ts Added a new sidebar link for "Datastore structure" under the "Types of nodes" section.
how-to-guides/celestia-node-store-structure.md Added a new section detailing the Celestia node's datastore structure and its contents.

Assessment against linked issues

Objective Addressed Explanation
Document the node's datastore and its contents (#1341)

Possibly related PRs

Suggested labels

unreleased-node, P0

Suggested reviewers

  • jcstein

Poem

In data stores, where bytes do rest,
A node's structure, we now attest.
With directories neat, and files so fine,
Celestia's secrets in clear design.
A change within our docs we see,
Clarity and structure, as it should be.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@jcstein
Copy link
Member

jcstein commented Jun 17, 2024

@coderabbitai review

Copy link
Contributor

coderabbitai bot commented Jun 17, 2024

Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@jcstein jcstein added the documentation Improvements or additions to documentation label Jun 17, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Outside diff range and nitpick comments (1)
nodes/celestia-node-store-structure.md (1)

34-34: Ensure the file ends with a single newline character.

Please add a newline at the end of the file to comply with Markdown best practices.

Tools
Markdownlint

34-34: null (MD047, single-trailing-newline)
Files should end with a single newline character

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 0643b2f and 8d93e5d.

Files selected for processing (2)
  • .vitepress/config.ts (1 hunks)
  • nodes/celestia-node-store-structure.md (1 hunks)
Files skipped from review due to trivial changes (1)
  • .vitepress/config.ts
Additional context used
LanguageTool
nodes/celestia-node-store-structure.md

[uncategorized] ~14-~14: Loose punctuation mark. (UNLIKELY_OPENING_PUNCTUATION)
Context: ...tained in the node's store: - /blocks: This directory stores blocks. Each file...


[uncategorized] ~15-~15: “It’s” is the contracted form of “it is”. Did you mean to use the possessive determiner “its”? (AI_HYDRA_LEO_SCY_IT_S)
Context: ...esents a block on Celestia and contains it's associated data. - /data: This direc...


[uncategorized] ~17-~17: Loose punctuation mark. (UNLIKELY_OPENING_PUNCTUATION)
Context: ...ontains it's associated data. - /data: This directory contains the block heade...


[uncategorized] ~19-~19: Loose punctuation mark. (UNLIKELY_OPENING_PUNCTUATION)
Context: ...ins the block header files. - /config: This directory stores the files used to...


[uncategorized] ~22-~22: Loose punctuation mark. (UNLIKELY_OPENING_PUNCTUATION)
Context: ...ing configuration files: - app.toml: The configuration file to define applic...


[uncategorized] ~24-~24: Loose punctuation mark. (UNLIKELY_OPENING_PUNCTUATION)
Context: ...d performance tuning. - client.toml: used to configure parameters for client...


[uncategorized] ~26-~26: Loose punctuation mark. (UNLIKELY_OPENING_PUNCTUATION)
Context: ... endpoints or timeouts. - config.toml: This is the node's primary configuratio...


[uncategorized] ~29-~29: Loose punctuation mark. (UNLIKELY_OPENING_PUNCTUATION)
Context: ...the node's identity. - inverted_index: This file stores indexed data that can ...


[uncategorized] ~29-~29: Possible missing preposition found. (AI_HYDRA_LEO_MISSING_TO)
Context: ...le stores indexed data that can be used make blockchain queries. - /keys: This di...


[uncategorized] ~31-~31: Loose punctuation mark. (UNLIKELY_OPENING_PUNCTUATION)
Context: ...used make blockchain queries. - /keys: This directory stores the cryptographic...


[uncategorized] ~33-~33: Loose punctuation mark. (UNLIKELY_OPENING_PUNCTUATION)
Context: ...ed to operate the node. - /transients: This directory stores temporary data su...

Markdownlint
nodes/celestia-node-store-structure.md

34-34: null (MD047, single-trailing-newline)
Files should end with a single newline character

nodes/celestia-node-store-structure.md Outdated Show resolved Hide resolved
nodes/celestia-node-store-structure.md Outdated Show resolved Hide resolved
nodes/celestia-node-store-structure.md Outdated Show resolved Hide resolved
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
- `client.toml`: used to configure parameters for client interactions like API endpoints or timeouts.

- `config.toml`: This is the node's primary configuration file.
It defines the node's core settings such as the network parameters and the node's identity.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"and the node's identity" is this true @renaynay ?

@jcstein jcstein requested review from renaynay June 17, 2024 19:50
Copy link
Collaborator

@distractedm1nd distractedm1nd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, thanks for this PR, but there are some confusing errors...

There is no app.toml or client.toml, or /config. Was this PR AI generated?

@Mackenzie-OO7
Copy link
Contributor Author

Hey, thanks for this PR, but there are some confusing errors...

There is no app.toml or client.toml, or /config. Was this PR AI generated?

Hi, I followed the draft in the linked issue as a guide. Is there somewhere else I could reference?

@jcstein
Copy link
Member

jcstein commented Jun 20, 2024

draft in the linked issue as a guide

yeah, I remember writing this but don't know where the extra directories and files are from. Are there more with full and bridge nodes? @Mackenzie-OO7 you can figure out the node store structure by initializing each node type celestia bridge init etc. and looking at the node store created. here's celestia light for example:

$ tree .celestia-light
.celestia-light
├── config.toml
├── data
│   ├── 000002.vlog
│   ├── 00022.mem
│   ├── 001174.sst
│   ├── DISCARD
│   ├── KEYREGISTRY
│   ├── LOCK
│   └── MANIFEST
└── keys
    ├── NJ3XILLTMVRXEZLUFZVHO5A
    ├── OAZHALLLMV4Q
    └── keyring-test
        ├── 968218...32d7c.address
        └── my_celes_key.info

4 directories, 129 files

@Mackenzie-OO7
Copy link
Contributor Author

yeah, I remember writing this but don't know where the extra directories and files are from. Are there more with full and bridge nodes? @Mackenzie-OO7 you can figure out the node store structure by initializing each node type celestia bridge init etc. and looking at the node store created. here's celestia light for example:

Ohh. Thanks!😅

@Mackenzie-OO7
Copy link
Contributor Author

Hi @jcstein could you explain what DISCARD, KEYREGISTRY, and MANIFEST store? I haven't been able to figure those out. Thanks!

@jcstein
Copy link
Member

jcstein commented Jul 24, 2024

hey, @distractedm1nd could you please help summarize what these are? thanks!

@distractedm1nd
Copy link
Collaborator

Belongs to badger, I have a general idea of what LOCK does but not the others - but i don't recommend describing these any further than "Various files belonging to Node LSM storage system"

@jcstein
Copy link
Member

jcstein commented Jul 24, 2024

Awesome, thanks @distractedm1nd. let's use "Various files belonging to Node LSM storage system" for all of those @Mackenzie-OO7

@jcstein
Copy link
Member

jcstein commented Aug 8, 2024

Kind bump here @Mackenzie-OO7 !

Awesome, thanks @distractedm1nd. let's use "Various files belonging to Node LSM storage system" for all of those @Mackenzie-OO7

@Mackenzie-OO7 Mackenzie-OO7 marked this pull request as ready for review August 14, 2024 09:36
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 8d93e5d and 838739e.

Files selected for processing (2)
  • .vitepress/config.ts (1 hunks)
  • nodes/celestia-node-store-structure.md (1 hunks)
Additional context used
LanguageTool
nodes/celestia-node-store-structure.md

[uncategorized] ~14-~14: Loose punctuation mark.
Context: ...les found in the datastore: - /blocks: This directory stores blocks. Each file...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~15-~15: Possible missing comma found.
Context: ... node's datastore does not contain this directory as light nodes do not store blocks. - ...

(AI_HYDRA_LEO_MISSING_COMMA)


[uncategorized] ~17-~17: Loose punctuation mark.
Context: ...ht nodes do not store blocks. - /data: This directory contains block headers a...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~19-~19: Loose punctuation mark.
Context: ...KEYREGISTRY, andMANIFEST. - /index`: This directory stores the index files t...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~19-~19: Possible missing comma found.
Context: ...ndex files that handle mapping specific keys such as block heights, to the correspon...

(AI_HYDRA_LEO_MISSING_COMMA)


[uncategorized] ~21-~21: Loose punctuation mark.
Context: ...t perform indexing. - /inverted_index: This directory stores the inverted inde...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~23-~23: Loose punctuation mark.
Context: ...not contain this directory. - /keys: This directory stores the cryptographic...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~25-~25: Loose punctuation mark.
Context: ...ed to operate the node. - /transients: This directory contains temporary data ...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~28-~28: Loose punctuation mark.
Context: ...anent blockchain state. - config.toml: This is the node's primary configuratio...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~28-~28: Possible missing comma found.
Context: ...ration file. It defines the node's core settings such as the network parameters.

(AI_HYDRA_LEO_MISSING_COMMA)

Additional comments not posted (2)
nodes/celestia-node-store-structure.md (1)

2-2: Correct grammar in the description.

Ensure grammatical accuracy by using "its" for possessive form.

.vitepress/config.ts (1)

459-462: Sidebar entry addition looks good.

The new sidebar entry for "Datastore structure" is correctly added and consistent with the existing structure.

nodes/celestia-node-store-structure.md Outdated Show resolved Hide resolved
nodes/celestia-node-store-structure.md Outdated Show resolved Hide resolved
nodes/celestia-node-store-structure.md Outdated Show resolved Hide resolved
nodes/celestia-node-store-structure.md Outdated Show resolved Hide resolved
nodes/celestia-node-store-structure.md Outdated Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range, codebase verification and nitpick comments (8)
nodes/celestia-node-store-structure.md (8)

14-14: Add a colon for clarity.

Add a colon after "datastore" for better readability.

- The following are the directories and files found in the datastore
+ The following are the directories and files found in the datastore:
Tools
LanguageTool

[uncategorized] ~14-~14: Loose punctuation mark.
Context: ...les found in the datastore: - /blocks: This directory stores blocks. Each file...

(UNLIKELY_OPENING_PUNCTUATION)


14-15: Ensure consistency in punctuation.

Ensure consistent use of punctuation at the end of each bullet point. Consider adding a period at the end of the sentence for consistency.

- represents a block on Celestia and contains its associated data. However, the light node's datastore does not contain this directory, as light nodes do not store blocks
+ represents a block on Celestia and contains its associated data. However, the light node's datastore does not contain this directory, as light nodes do not store blocks.
Tools
LanguageTool

[uncategorized] ~14-~14: Loose punctuation mark.
Context: ...les found in the datastore: - /blocks: This directory stores blocks. Each file...

(UNLIKELY_OPENING_PUNCTUATION)


17-17: Add a comma for clarity.

Add a comma before "such as" for better readability.

- and various files belonging to node LSM storage system such as `DISCARD`, `KEYREGISTRY`, and `MANIFEST`.
+ and various files belonging to node LSM storage system, such as `DISCARD`, `KEYREGISTRY`, and `MANIFEST`.
Tools
LanguageTool

[uncategorized] ~17-~17: Loose punctuation mark.
Context: ...ht nodes do not store blocks. - /data: This directory contains block headers a...

(UNLIKELY_OPENING_PUNCTUATION)


19-19: Add a comma for clarity.

Add a comma before "as they do not perform indexing" for better readability.

- does not include this directory, as they do not perform indexing.
+ does not include this directory, as they do not perform indexing.
Tools
LanguageTool

[uncategorized] ~19-~19: Loose punctuation mark.
Context: ...EYREGISTRY, and MANIFEST. - /index`: This directory stores the index files t...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~19-~19: Possible missing comma found.
Context: ...ndex files that handle mapping specific keys such as block heights, to the correspon...

(AI_HYDRA_LEO_MISSING_COMMA)


21-21: Add a comma for clarity.

Add a comma before "such as" for better readability.

- and various files belonging to node LSM storage system, such as `DISCARD`, `KEYREGISTRY`, `LOCK`, and `MANIFEST`.
+ and various files belonging to node LSM storage system, such as `DISCARD`, `KEYREGISTRY`, `LOCK`, and `MANIFEST`.
Tools
LanguageTool

[uncategorized] ~21-~21: Loose punctuation mark.
Context: ...t perform indexing. - /inverted_index: This directory stores the inverted inde...

(UNLIKELY_OPENING_PUNCTUATION)


23-23: Ensure consistent punctuation.

Ensure consistent use of punctuation at the end of each bullet point. Consider adding a period at the end of the sentence for consistency.

- This directory stores the cryptographic key pairs that are used to operate the node
+ This directory stores the cryptographic key pairs that are used to operate the node.
Tools
LanguageTool

[uncategorized] ~23-~23: Loose punctuation mark.
Context: ...not contain this directory. - /keys: This directory stores the cryptographic...

(UNLIKELY_OPENING_PUNCTUATION)


25-25: Ensure consistent punctuation.

Ensure consistent use of punctuation at the end of each bullet point. Consider adding a period at the end of the sentence for consistency.

- that are used while the node is operating, but are not a part of the permanent blockchain state
+ that are used while the node is operating, but are not a part of the permanent blockchain state.
Tools
LanguageTool

[uncategorized] ~25-~25: Loose punctuation mark.
Context: ...ed to operate the node. - /transients: This directory contains temporary data ...

(UNLIKELY_OPENING_PUNCTUATION)


28-28: Ensure consistent punctuation.

Ensure consistent use of punctuation at the end of each bullet point. Consider adding a period at the end of the sentence for consistency.

- It defines the node's core settings, such as the network parameters
+ It defines the node's core settings, such as the network parameters.
Tools
LanguageTool

[uncategorized] ~28-~28: Loose punctuation mark.
Context: ...anent blockchain state. - config.toml: This is the node's primary configuratio...

(UNLIKELY_OPENING_PUNCTUATION)

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 838739e and 192f096.

Files selected for processing (1)
  • nodes/celestia-node-store-structure.md (1 hunks)
Additional context used
LanguageTool
nodes/celestia-node-store-structure.md

[uncategorized] ~14-~14: Loose punctuation mark.
Context: ...les found in the datastore: - /blocks: This directory stores blocks. Each file...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~17-~17: Loose punctuation mark.
Context: ...ht nodes do not store blocks. - /data: This directory contains block headers a...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~19-~19: Loose punctuation mark.
Context: ...EYREGISTRY, and MANIFEST. - /index`: This directory stores the index files t...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~19-~19: Possible missing comma found.
Context: ...ndex files that handle mapping specific keys such as block heights, to the correspon...

(AI_HYDRA_LEO_MISSING_COMMA)


[uncategorized] ~21-~21: Loose punctuation mark.
Context: ...t perform indexing. - /inverted_index: This directory stores the inverted inde...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~23-~23: Loose punctuation mark.
Context: ...not contain this directory. - /keys: This directory stores the cryptographic...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~25-~25: Loose punctuation mark.
Context: ...ed to operate the node. - /transients: This directory contains temporary data ...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~28-~28: Loose punctuation mark.
Context: ...anent blockchain state. - config.toml: This is the node's primary configuratio...

(UNLIKELY_OPENING_PUNCTUATION)

Additional comments not posted (2)
nodes/celestia-node-store-structure.md (2)

1-3: LGTM!

The description is clear and grammatically correct.


5-10: LGTM!

The introduction provides a clear and concise overview of the node's datastore.

nodes/celestia-node-store-structure.md Outdated Show resolved Hide resolved
nodes/celestia-node-store-structure.md Outdated Show resolved Hide resolved
@jcstein jcstein mentioned this pull request Oct 1, 2024
@jcstein
Copy link
Member

jcstein commented Oct 23, 2024

let's close this and rework it into the new structure. wdyt? also after reviewing, I think a lot of what is maybe AI generated here needs to go. many of these aren't in my node store...

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 4f59964 and 069cab7.

📒 Files selected for processing (1)
  • .vitepress/config.ts (1 hunks)
🧰 Additional context used

Comment on lines 470 to 473
{
text: "Datastore structure",
link: "nodes/celestia-node-store-structure",
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Update the link path to follow documentation conventions.

The link path nodes/celestia-node-store-structure doesn't follow the established pattern used throughout the documentation (which typically starts with /how-to-guides/). This could lead to broken links.

Apply this diff to align with the documentation structure:

                {
                  text: "Datastore structure",
-                  link: "nodes/celestia-node-store-structure",
+                  link: "/how-to-guides/celestia-node-store-structure",
                },
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
{
text: "Datastore structure",
link: "nodes/celestia-node-store-structure",
},
{
text: "Datastore structure",
link: "/how-to-guides/celestia-node-store-structure",
},

@jcstein jcstein mentioned this pull request Oct 23, 2024
@Mackenzie-OO7
Copy link
Contributor Author

let's close this and rework it into the new structure. wdyt? also after reviewing, I think a lot of what is maybe AI generated here needs to go. many of these aren't in my node store...

so initially I kinda mixed up the files & directories that I listed with the ones in celestia-app. but I changed that already, and now the directories listed here are all in the celestia-node store.

here's celestia light for example:

$ tree .celestia-light
.celestia-light
├── config.toml
├── data
│   ├── 000002.vlog
│   ├── 00022.mem
│   ├── 001174.sst
│   ├── DISCARD
│   ├── KEYREGISTRY
│   ├── LOCK
│   └── MANIFEST
└── keys
    ├── NJ3XILLTMVRXEZLUFZVHO5A
    ├── OAZHALLLMV4Q
    └── keyring-test
        ├── 968218...32d7c.address
        └── my_celes_key.info

4 directories, 129 files

The directories you don't have here are in the bridge and full nodes

@jcstein
Copy link
Member

jcstein commented Oct 24, 2024

makes sense! thank you for clarifying

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Outside diff range and nitpick comments (2)
how-to-guides/celestia-node-store-structure.md (2)

7-10: Consider mentioning node types in the introduction

Since the datastore structure varies between light, bridge, and full nodes, it would be helpful to mention this upfront in the introduction.

 The node's datastore refers to the storage structure
 used to manage the data that supports the node's operation.
 It consists of directories and files that contain the node's state,
-configuration, and other information relevant to the node.
+configuration, and other information relevant to the node. The structure
+varies depending on the node type (light, bridge, or full node).

14-15: Improve clarity of blocks directory description

Consider adding more context about why light nodes don't store blocks.

-represents a block on Celestia and contains its associated data. This directory is present in the datastore for bridge and full nodes but not light nodes, as light nodes do not store blocks.
+represents a block on Celestia and contains its associated data. This directory is present in the datastore for bridge and full nodes but not light nodes, as light nodes only verify headers and don't need to store full blocks.
🧰 Tools
🪛 LanguageTool

[uncategorized] ~14-~14: Loose punctuation mark.
Context: ...les found in the datastore: - /blocks: This directory stores blocks. Each file...

(UNLIKELY_OPENING_PUNCTUATION)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 069cab7 and 31c94da.

📒 Files selected for processing (2)
  • .vitepress/config.ts (1 hunks)
  • how-to-guides/celestia-node-store-structure.md (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • .vitepress/config.ts
🧰 Additional context used
🪛 LanguageTool
how-to-guides/celestia-node-store-structure.md

[uncategorized] ~14-~14: Loose punctuation mark.
Context: ...les found in the datastore: - /blocks: This directory stores blocks. Each file...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~17-~17: Loose punctuation mark.
Context: ...ht nodes do not store blocks. - /data: This directory contains block headers a...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~19-~19: Loose punctuation mark.
Context: ...EYREGISTRY, and MANIFEST. - /index`: This directory stores the index files t...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~21-~21: Loose punctuation mark.
Context: ...t perform indexing. - /inverted_index: This directory stores the inverted inde...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~23-~23: Loose punctuation mark.
Context: ...not contain this directory. - /keys: This directory stores the cryptographic...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~25-~25: Loose punctuation mark.
Context: ...ed to operate the node. - /transients: This directory contains temporary data ...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~28-~28: Loose punctuation mark.
Context: ...anent blockchain state. - config.toml: This is the node's primary configuratio...

(UNLIKELY_OPENING_PUNCTUATION)

🔇 Additional comments (1)
how-to-guides/celestia-node-store-structure.md (1)

12-28: Verify directory structure completeness

As suggested by @jcstein in the PR discussion, it would be valuable to verify this documentation against the actual directory structure created by initializing different node types (e.g., celestia bridge init, celestia light init).

🧰 Tools
🪛 LanguageTool

[uncategorized] ~14-~14: Loose punctuation mark.
Context: ...les found in the datastore: - /blocks: This directory stores blocks. Each file...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~17-~17: Loose punctuation mark.
Context: ...ht nodes do not store blocks. - /data: This directory contains block headers a...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~19-~19: Loose punctuation mark.
Context: ...EYREGISTRY, and MANIFEST. - /index`: This directory stores the index files t...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~21-~21: Loose punctuation mark.
Context: ...t perform indexing. - /inverted_index: This directory stores the inverted inde...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~23-~23: Loose punctuation mark.
Context: ...not contain this directory. - /keys: This directory stores the cryptographic...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~25-~25: Loose punctuation mark.
Context: ...ed to operate the node. - /transients: This directory contains temporary data ...

(UNLIKELY_OPENING_PUNCTUATION)


[uncategorized] ~28-~28: Loose punctuation mark.
Context: ...anent blockchain state. - config.toml: This is the node's primary configuratio...

(UNLIKELY_OPENING_PUNCTUATION)


- `/index`: This directory stores the index files that handle mapping specific keys such as block heights, to the corresponding data. Similar to `/blocks`, the light node's datastore does not include this directory, as they do not perform indexing.

- `/inverted_index`: This directory stores the inverted index files used for mapping queries to the corresponding data location, and various files belonging to node LSM storage system, such as `DISCARD`, `KEYREGISTRY`, `LOCK`, and `MANIFEST`. The light node's datastore does not contain this directory.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Remove redundant LSM file listing

The LSM files are already mentioned in the /data directory description.

-This directory stores the inverted index files used for mapping queries to the corresponding data location, and various files belonging to node LSM storage system, such as `DISCARD`, `KEYREGISTRY`, `LOCK`, and `MANIFEST`. The light node's datastore does not contain this directory.
+This directory stores the inverted index files used for mapping queries to the corresponding data location, along with associated LSM storage system files. The light node's datastore does not contain this directory.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- `/inverted_index`: This directory stores the inverted index files used for mapping queries to the corresponding data location, and various files belonging to node LSM storage system, such as `DISCARD`, `KEYREGISTRY`, `LOCK`, and `MANIFEST`. The light node's datastore does not contain this directory.
- `/inverted_index`: This directory stores the inverted index files used for mapping queries to the corresponding data location, along with associated LSM storage system files. The light node's datastore does not contain this directory.
🧰 Tools
🪛 LanguageTool

[uncategorized] ~21-~21: Loose punctuation mark.
Context: ...t perform indexing. - /inverted_index: This directory stores the inverted inde...

(UNLIKELY_OPENING_PUNCTUATION)

- `/transients`: This directory contains temporary data such as cache files
that are used while the node is operating, but are not a part of the permanent blockchain state.

- `config.toml`: This is the node's primary configuration file. It defines the node's core settings, such as the network parameters.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Clarify config.toml location and purpose

Based on the PR discussion, there was some confusion about configuration files. Let's be more specific about the location and purpose of config.toml.

-`config.toml`: This is the node's primary configuration file. It defines the node's core settings, such as the network parameters.
+`config.toml`: Located in the node's root directory, this is the primary configuration file that defines core settings such as network parameters, P2P configuration, and API endpoints. This file is generated during node initialization.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- `config.toml`: This is the node's primary configuration file. It defines the node's core settings, such as the network parameters.
- `config.toml`: Located in the node's root directory, this is the primary configuration file that defines core settings such as network parameters, P2P configuration, and API endpoints. This file is generated during node initialization.
🧰 Tools
🪛 LanguageTool

[uncategorized] ~28-~28: Loose punctuation mark.
Context: ...anent blockchain state. - config.toml: This is the node's primary configuratio...

(UNLIKELY_OPENING_PUNCTUATION)

- `/blocks`: This directory stores blocks. Each file contained in this directory
represents a block on Celestia and contains its associated data. This directory is present in the datastore for bridge and full nodes but not light nodes, as light nodes do not store blocks.

- `/data`: This directory contains block headers and various files belonging to node log-structured merge (LSM) storage system such as `DISCARD`, `KEYREGISTRY`, and `MANIFEST`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Clarify LSM storage system files

Based on the discussion with @distractedm1nd, we should keep the LSM storage system description general.

-This directory contains block headers and various files belonging to node log-structured merge (LSM) storage system such as `DISCARD`, `KEYREGISTRY`, and `MANIFEST`.
+This directory contains block headers and various files belonging to the node's log-structured merge (LSM) storage system. The LSM files (such as `DISCARD`, `KEYREGISTRY`, and `MANIFEST`) manage the efficient storage and retrieval of data.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- `/data`: This directory contains block headers and various files belonging to node log-structured merge (LSM) storage system such as `DISCARD`, `KEYREGISTRY`, and `MANIFEST`.
- `/data`: This directory contains block headers and various files belonging to the node's log-structured merge (LSM) storage system. The LSM files (such as `DISCARD`, `KEYREGISTRY`, and `MANIFEST`) manage the efficient storage and retrieval of data.
🧰 Tools
🪛 LanguageTool

[uncategorized] ~17-~17: Loose punctuation mark.
Context: ...ht nodes do not store blocks. - /data: This directory contains block headers a...

(UNLIKELY_OPENING_PUNCTUATION)

@jcstein jcstein reopened this Oct 24, 2024
@jcstein
Copy link
Member

jcstein commented Oct 24, 2024

didn't fully resolve in that PR! 😅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

docs: Document the node's store structure clearly
3 participants