From 82c6e8b47dbf4242a082ff4fdf67dcb33c496c0a Mon Sep 17 00:00:00 2001 From: hasanza Date: Tue, 2 Jul 2024 15:15:03 +0500 Subject: [PATCH 1/5] add intro and installation guide --- README.md | 78 +++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 56 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index e9c10e7..f2c8a4a 100644 --- a/README.md +++ b/README.md @@ -1,22 +1,56 @@ -# Quicksync-rs - -How to use: -``` -quicksync-rs help -``` - -Development: -``` -cargo run -- help -``` - -## Exit codes -- `0` - all good -- `1` - failed to download archive within max retries (any reason) -- `2` - cannot unpack archive: not enough disk space -- `3` - cannot unpack archive: any other reason -- `4` - invalid checksum of db -- `5` - cannot verify checksum for some reason -- `6` - cannot create a backup file -- `7` - invalid checksum of archive -- `8` - cannot validate archive checksum \ No newline at end of file +# quicksync-rs + +When a new node joins the Spacemesh network, it must first get up to speed with the rest of its peers. This process is referred to as "syncing" and is a pre-requisite to running a full or a smeshing node. Historically, it has been difficult for smeshers to successfully sync their nodes owing to how time-consuming the syncing process is. Primarily, syncing includes downloading and independently verifying all blocks, transactions, ATXs, along with some other data, and reconstructing the current state from scratch. Naturally, this took the average smesher a lot of time. As such, in response to the growing difficulty of syncing a fresh node, we have prepared a way to speed up the syncing process. Introducing, Quicksync. + +With Quicksync, instead of performing all of the syncing actions as stated above and calculating the network state from genesis, one just needs to download the current state from a trusted peer like the Spacemesh dev team or some other node. While this runs contrary to the web3 philosophy of "Don't trust, verify", we believe that this could be a choice some smeshers may be interested in given the high rate of trouble with syncing. Moreover, nothing precludes a smesher from verifying this state in the background once it is downloaded. + +The state (also called an archive) that is downloaded is in the form of a state.sql file and can either be downloaded automatically using Smapp, or manually by using the `quicksync-rs` utility. + +Instructions for using `quicksync-rs` to download the state are given below. + +## Windows + +1. Download the latest release of the `quicksync-windows-vX.X.X.zip` file from the GitHub releases section. +2. Extract the `quicksync.exe` file from the zip file downloaded in step 1. +3. Move the `quicksync.exe` file to your `go-spacemesh` folder. +4. If you see a `state.sql` file in this folder, delete it. Otherwise, continue to step 5. +5. Open the Windows Powershell in this folder. You can do this by holding the "shift" key, right-clicking, and selecting the "Open PowerShell here" option. +6. Inside the powershell, type `.\quicksync.exe --help` and press enter. This will show you the available options. +7. We want to download the state database. Type `.\quicksync.exe download --node-data `. Note that the node data folder can be identified by the other folders and files it contains such as `bootstrap/`, `p2p/`, `genesis.json`, etc. A lot of smeshers label the node data folder as `sm_data` or just `data`. In the official Spacemesh docs and guides, this folder has been labelled as `node_data`. +8. Wait for the process to complete. The `quicksync-rs` utility will download, unzip, and verify the downloaded state. +9. Your node data folder should now have the latest `state.sql` file. + +## Linux + +1. Download the latest release of the `quicksync-linux-vX.X.X.zip` file from the GitHub releases section. +2. Extract the `quicksync` file from the zip file downloaded in step 1. +3. Make the `quicksync` file executable by using this CLI command: `chmod +x quicksync`. Now you have the `quicksync` executable. +4. Move the `quicksync` executable to your `go-spacemesh` folder. +5. Delete the existing `state.sql` file from the node data directory. +6. Open a terminal in the `go-spacemesh` directory where the `quicksync` executable is, and run this command: `./quicksync download --node-data `. +7. Wait for the process to complete. The `quicksync-rs` utility will download, unzip, and verify the downloaded state. +8. Your node data folder should now have the latest `state.sql` file. + +## Exit Codes + +Listed below are the exit codes and what they mean: + +- `0` - All good. +- `1` - Failed to download archive within max retries (any reason). +- `2` - Cannot unpack archive: not enough disk space. +- `3` - Cannot unpack archive: any other reason. +- `4` - Invalid checksum of downloaded `state.sql`. +- `5` - Cannot verify checksum for some reason. +- `6` - Cannot create a backup file. +- `7` - Invalid checksum of archive. +- `8` - Cannot validate archive checksum. + +## Commands + +The list of available commands for the `quicksync` utility is presented below. Note that these commands are for Linux. Simply, Change `./quicksync` to `.\quicksync.exe` For the Windows commands. + +- `./quicksync download`: Downloads the latest `state.sql` file. +- `./quicksync check`: Checks if the current `state.sql` is up to date or not. +- `./quicksync help`: Displays all operations that `quicksync` can perform. +- `./quicksync --version`: Displays the quicksync version. +- `cargo run -- help`: Displays helpful commands for running the package. Relevant for developers. From 40b4cde1b3e41f60c070b836f9f89203bfb1acdc Mon Sep 17 00:00:00 2001 From: hasanza <49759922+hasanza@users.noreply.github.com> Date: Tue, 2 Jul 2024 18:28:35 +0500 Subject: [PATCH 2/5] Update README.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Bartosz Różański --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index f2c8a4a..efa6d53 100644 --- a/README.md +++ b/README.md @@ -50,7 +50,7 @@ Listed below are the exit codes and what they mean: The list of available commands for the `quicksync` utility is presented below. Note that these commands are for Linux. Simply, Change `./quicksync` to `.\quicksync.exe` For the Windows commands. - `./quicksync download`: Downloads the latest `state.sql` file. -- `./quicksync check`: Checks if the current `state.sql` is up to date or not. +- `./quicksync check`: Checks if the current `state.sql` is up to date. - `./quicksync help`: Displays all operations that `quicksync` can perform. - `./quicksync --version`: Displays the quicksync version. - `cargo run -- help`: Displays helpful commands for running the package. Relevant for developers. From 76aea120b0894cd23309fa61639099899280430a Mon Sep 17 00:00:00 2001 From: hasanza Date: Wed, 3 Jul 2024 15:21:00 +0500 Subject: [PATCH 3/5] Added exact paths --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index f2c8a4a..2345dff 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ Instructions for using `quicksync-rs` to download the state are given below. 4. If you see a `state.sql` file in this folder, delete it. Otherwise, continue to step 5. 5. Open the Windows Powershell in this folder. You can do this by holding the "shift" key, right-clicking, and selecting the "Open PowerShell here" option. 6. Inside the powershell, type `.\quicksync.exe --help` and press enter. This will show you the available options. -7. We want to download the state database. Type `.\quicksync.exe download --node-data `. Note that the node data folder can be identified by the other folders and files it contains such as `bootstrap/`, `p2p/`, `genesis.json`, etc. A lot of smeshers label the node data folder as `sm_data` or just `data`. In the official Spacemesh docs and guides, this folder has been labelled as `node_data`. +7. We want to download the state database. Type `.\quicksync.exe download --node-data `. By default, the node data folder is located in the `C:\Users\{USERNAME}\AppData\Roaming\Spacemesh` directory and can be identified by the other folders and files it contains such as `bootstrap/`, `p2p/`, `genesis.json`, etc. A lot of smeshers label the node data folder as `sm_data` or just `data`. In the official Spacemesh docs and guides, this folder has been labelled as `node_data`. 8. Wait for the process to complete. The `quicksync-rs` utility will download, unzip, and verify the downloaded state. 9. Your node data folder should now have the latest `state.sql` file. @@ -27,7 +27,7 @@ Instructions for using `quicksync-rs` to download the state are given below. 3. Make the `quicksync` file executable by using this CLI command: `chmod +x quicksync`. Now you have the `quicksync` executable. 4. Move the `quicksync` executable to your `go-spacemesh` folder. 5. Delete the existing `state.sql` file from the node data directory. -6. Open a terminal in the `go-spacemesh` directory where the `quicksync` executable is, and run this command: `./quicksync download --node-data `. +6. Open a terminal in the `go-spacemesh` directory where the `quicksync` executable is, and run this command: `./quicksync download --node-data `. By default, the node data folder is located in `~/.config/Spacemesh`. 7. Wait for the process to complete. The `quicksync-rs` utility will download, unzip, and verify the downloaded state. 8. Your node data folder should now have the latest `state.sql` file. From 16190bcdbb5562092ec30b3a6bb182d9254fde03 Mon Sep 17 00:00:00 2001 From: hasanza Date: Wed, 3 Jul 2024 18:32:59 +0500 Subject: [PATCH 4/5] further simplified instructions --- README.md | 35 +++++++++++++++++++++++------------ 1 file changed, 23 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index c1ebe35..9677072 100644 --- a/README.md +++ b/README.md @@ -6,28 +6,39 @@ With Quicksync, instead of performing all of the syncing actions as stated above The state (also called an archive) that is downloaded is in the form of a state.sql file and can either be downloaded automatically using Smapp, or manually by using the `quicksync-rs` utility. -Instructions for using `quicksync-rs` to download the state are given below. +Instructions for using `quicksync-rs` to download the latest state are given below. Note that if you use the latest version of Smapp, it will automatically offer to use quicksync to fetch the latest state. ## Windows -1. Download the latest release of the `quicksync-windows-vX.X.X.zip` file from the GitHub releases section. -2. Extract the `quicksync.exe` file from the zip file downloaded in step 1. -3. Move the `quicksync.exe` file to your `go-spacemesh` folder. -4. If you see a `state.sql` file in this folder, delete it. Otherwise, continue to step 5. -5. Open the Windows Powershell in this folder. You can do this by holding the "shift" key, right-clicking, and selecting the "Open PowerShell here" option. -6. Inside the powershell, type `.\quicksync.exe --help` and press enter. This will show you the available options. -7. We want to download the state database. Type `.\quicksync.exe download --node-data `. By default, the node data folder is located in the `C:\Users\{USERNAME}\AppData\Roaming\Spacemesh` directory and can be identified by the other folders and files it contains such as `bootstrap/`, `p2p/`, `genesis.json`, etc. A lot of smeshers label the node data folder as `sm_data` or just `data`. In the official Spacemesh docs and guides, this folder has been labelled as `node_data`. +1. Download the latest release of `quicksync-windows-vX.X.X.zip` from the GitHub releases section. +2. Extract `quicksync.exe` from the zip file downloaded in step 1. +3. Move `quicksync.exe` to your Spacemesh folder. By default, this folder is located at: `C:\Users\{USERNAME}\AppData\Roaming\Spacemesh`. +4. If you see a `state.sql` file in your node data folder (located inside the `Spacemesh` directory and named `node-data` by default), delete it. Otherwise, continue to step 5. +5. Open a Windows Powershell terminal in the `Spacemesh` directory. You can do this by holding the "shift" key, right-clicking, and selecting the "Open Powershell here" option. +6. Inside the Powershell, type `.\quicksync.exe --help` and press enter. This will show you the available options. +7. We want to download the state database. Type `.\quicksync.exe download --node-data .\node-data`. Here, `.\node-data` is the path to the node data folder. 8. Wait for the process to complete. The `quicksync-rs` utility will download, unzip, and verify the downloaded state. 9. Your node data folder should now have the latest `state.sql` file. ## Linux -1. Download the latest release of the `quicksync-linux-vX.X.X.zip` file from the GitHub releases section. +1. Download the latest release of `quicksync-linux-vX.X.X.zip` from the GitHub releases section. 2. Extract the `quicksync` file from the zip file downloaded in step 1. 3. Make the `quicksync` file executable by using this CLI command: `chmod +x quicksync`. Now you have the `quicksync` executable. -4. Move the `quicksync` executable to your `go-spacemesh` folder. -5. Delete the existing `state.sql` file from the node data directory. -6. Open a terminal in the `go-spacemesh` directory where the `quicksync` executable is, and run this command: `./quicksync download --node-data `. By default, the node data folder is located in `~/.config/Spacemesh`. +4. Move the `quicksync` executable to the `Spacemesh` directory ( located at `~/.config/Spacemesh` by default). +5. If you see a `state.sql` file in your node data folder (located inside the `Spacemesh` directory and named `node-data` by default), delete it. Otherwise, continue to step 6. +6. Open a terminal in the `Spacemesh` directory where the `quicksync` executable is, and run this command: `./quicksync download --node-data ./node-data`. Here, `./node-data` is the path to the node data folder. +7. Wait for the process to complete. The `quicksync-rs` utility will download, unzip, and verify the downloaded state. +8. Your node data folder should now have the latest `state.sql` file. + +## MacOS + +1. Download the latest release of `quicksync-macos-vX.X.X.zip` (or `quicksync-macos-arm64-vX.X.X.zip` if you have an M-series Mac) from the GitHub releases section. +2. Extract the `quicksync` file from the zip file downloaded in step 1. +3. Make the `quicksync` file executable by using this CLI command: `chmod +x quicksync`. Now you have the `quicksync` executable. +4. Move the `quicksync` executable to the `Spacemesh` directory. By default, this directory is located at `~/Library/Application\ Support/Spacemesh`. +5. If you see a `state.sql` file in your node data folder (located inside the `Spacemesh` directory and named `node-data` by default), delete it. Otherwise, continue to step 6. +6. Open a terminal in the `Spacemesh` directory where the `quicksync` executable is, and run this command: `./quicksync download --node-data ./node-data`. Here, `./node-data` is the path to the node data folder. 7. Wait for the process to complete. The `quicksync-rs` utility will download, unzip, and verify the downloaded state. 8. Your node data folder should now have the latest `state.sql` file. From fc3b18fe4933ff7af0f8138dd2b71d7a02bdca24 Mon Sep 17 00:00:00 2001 From: hasanza Date: Thu, 4 Jul 2024 21:58:58 +0500 Subject: [PATCH 5/5] fixed paths --- .vscode/settings.json | 6 ++++++ README.md | 20 ++++++++++---------- 2 files changed, 16 insertions(+), 10 deletions(-) create mode 100644 .vscode/settings.json diff --git a/.vscode/settings.json b/.vscode/settings.json new file mode 100644 index 0000000..cd6c0d1 --- /dev/null +++ b/.vscode/settings.json @@ -0,0 +1,6 @@ +{ + "cSpell.words": [ + "quicksync", + "spacemesh" + ] +} \ No newline at end of file diff --git a/README.md b/README.md index 9677072..62d237b 100644 --- a/README.md +++ b/README.md @@ -12,9 +12,9 @@ Instructions for using `quicksync-rs` to download the latest state are given bel 1. Download the latest release of `quicksync-windows-vX.X.X.zip` from the GitHub releases section. 2. Extract `quicksync.exe` from the zip file downloaded in step 1. -3. Move `quicksync.exe` to your Spacemesh folder. By default, this folder is located at: `C:\Users\{USERNAME}\AppData\Roaming\Spacemesh`. -4. If you see a `state.sql` file in your node data folder (located inside the `Spacemesh` directory and named `node-data` by default), delete it. Otherwise, continue to step 5. -5. Open a Windows Powershell terminal in the `Spacemesh` directory. You can do this by holding the "shift" key, right-clicking, and selecting the "Open Powershell here" option. +3. Move `quicksync.exe` to your `spacemesh` folder. By default, this folder is located at: `C:\Users\{USERNAME}\spacemesh`. +4. If you see a `state.sql` file in your node data folder (located inside the `spacemesh` directory and named `node-data` by default), delete it. Otherwise, continue to step 5. +5. Open a Windows Powershell terminal in the `spacemesh` directory where the `quicksync.exe` file is. You can do this by holding the "shift" key, right-clicking, and selecting the "Open Powershell here" option. 6. Inside the Powershell, type `.\quicksync.exe --help` and press enter. This will show you the available options. 7. We want to download the state database. Type `.\quicksync.exe download --node-data .\node-data`. Here, `.\node-data` is the path to the node data folder. 8. Wait for the process to complete. The `quicksync-rs` utility will download, unzip, and verify the downloaded state. @@ -25,20 +25,20 @@ Instructions for using `quicksync-rs` to download the latest state are given bel 1. Download the latest release of `quicksync-linux-vX.X.X.zip` from the GitHub releases section. 2. Extract the `quicksync` file from the zip file downloaded in step 1. 3. Make the `quicksync` file executable by using this CLI command: `chmod +x quicksync`. Now you have the `quicksync` executable. -4. Move the `quicksync` executable to the `Spacemesh` directory ( located at `~/.config/Spacemesh` by default). -5. If you see a `state.sql` file in your node data folder (located inside the `Spacemesh` directory and named `node-data` by default), delete it. Otherwise, continue to step 6. -6. Open a terminal in the `Spacemesh` directory where the `quicksync` executable is, and run this command: `./quicksync download --node-data ./node-data`. Here, `./node-data` is the path to the node data folder. +4. Move the `quicksync` executable to the `spacemesh` directory ( located at `~/spacemesh` by default). +5. If you see a `state.sql` file in your node data folder (located inside the `spacemesh` directory and named `node-data` by default), delete it. Otherwise, continue to step 6. +6. Open a terminal in the `spacemesh` directory where the `quicksync` executable is, and run this command: `./quicksync download --node-data ./node-data`. Here, `./node-data` is the path to the node data folder. 7. Wait for the process to complete. The `quicksync-rs` utility will download, unzip, and verify the downloaded state. 8. Your node data folder should now have the latest `state.sql` file. -## MacOS +## MacOS 1. Download the latest release of `quicksync-macos-vX.X.X.zip` (or `quicksync-macos-arm64-vX.X.X.zip` if you have an M-series Mac) from the GitHub releases section. 2. Extract the `quicksync` file from the zip file downloaded in step 1. 3. Make the `quicksync` file executable by using this CLI command: `chmod +x quicksync`. Now you have the `quicksync` executable. -4. Move the `quicksync` executable to the `Spacemesh` directory. By default, this directory is located at `~/Library/Application\ Support/Spacemesh`. -5. If you see a `state.sql` file in your node data folder (located inside the `Spacemesh` directory and named `node-data` by default), delete it. Otherwise, continue to step 6. -6. Open a terminal in the `Spacemesh` directory where the `quicksync` executable is, and run this command: `./quicksync download --node-data ./node-data`. Here, `./node-data` is the path to the node data folder. +4. Move the `quicksync` executable to the `spacemesh` directory. (located at `~/spacemesh` by default). +5. If you see a `state.sql` file in your node data folder (located inside the `spacemesh` directory and named `node-data` by default), delete it. Otherwise, continue to step 6. +6. Open a terminal in the `spacemesh` directory where the `quicksync` executable is, and run this command: `./quicksync download --node-data ./node-data`. Here, `./node-data` is the path to the node data folder. 7. Wait for the process to complete. The `quicksync-rs` utility will download, unzip, and verify the downloaded state. 8. Your node data folder should now have the latest `state.sql` file.