Merge pull request #327 from shachlanAmazon/docs

Improve and sort documentation.

.github/workflows/rust.yml

@@ -24,8 +24,8 @@ jobs:
             fail-fast: false
             matrix:
                 redis:
-                    - 6.2.4
-                    - 7.0.0
+                    - 6.2.13
+                    - 7.0.12
 
         steps:
             - uses: actions/checkout@v3

README.md

@@ -1,21 +1,40 @@
 # babushka
 
-## developer pre-requirements:
+Babushka (temporary name, final name pending) is a collection of open source Redis clients in various managed languages, based around shared logic written in Rust. We call the clients "wrappers", and the shared logic "core".
 
- - GCC
- - pkg-config
- - protobuf-compiler (protoc)
- - openssl
- - libssl-dev
+## Supported languages
+
+-   [node](./node/README.md).
+-   [python](./python/README.md)
+
+## Folder structure
+
+-   babushka-core - [the Rust core](./babushka-core/README.md). This code is compiled into binaries which are used by the various wrappers.
+-   benchmarks - [benchmarks for the wrappers](./benchmarks/README.md), and for the current leading Redis client in each language.
+-   csharp - [the C# wrapper](./csharp/README.md). currently not in a usable state.
+-   logger_core - [the logger we use](./logger_core/).
+-   node - [the NodeJS wrapper](./node/README.md).
+-   python - [the Python3 wrapper](./python/README.md)
+-   submodules - git submodules.
+-   utils - Utility scripts used in testing, building, and deploying.
+
+## development pre-requirements
+
+-   GCC
+-   pkg-config
+-   protobuf-compiler (protoc)
+-   openssl
+-   libssl-dev
+-   python3
 
 Installation for ubuntu:
-`sudo apt install -y gcc pkg-config protobuf-compiler openssl libssl-dev`
+`sudo apt install -y gcc pkg-config protobuf-compiler openssl libssl-dev python3`
 
-### git submodule -
+### git submodule
 
 run `git submodule update --init --recursive` on first usage, and on any update to the redis-rs fork.
 
-### rustup -
+### rustup
 
 https://rustup.rs/
 
@@ -27,6 +46,8 @@ after the instalation will show-up in the terminal steps to add rustup to the pa
 
 ### redis-server
 
+This is required for running tests and local benchmarks.
+
 https://redis.io/docs/getting-started/
 
 ```
@@ -42,56 +63,19 @@ sudo mkdir /etc/redis
 sudo cp src/redis-server src/redis-cli /usr/local/bin
 ```
 
-change {0} to the version you want, e.g. 7.0.7. version names are available here: http://download.redis.io/releases/
-recommended version - 7.0.7 ATM
+change {0} to the version you want, e.g. 7.0.12. version names are available here: http://download.redis.io/releases/
+recommended version - 7.0.12 ATM
 
 ### node 16 (or newer)
 
+This is required for the NodeJS wrapper, and for running benchmarks.
+
 ```
 curl -s https://deb.nodesource.com/setup_16.x | sudo bash
 apt-get install nodejs npm
 npm i -g npm@8
 ```
 
-## benchmarks:
-
-`benchmarks/install_and_test.sh` is the benchmark script we use to check performance. run `install_and_test.sh -h` to get the full list of available flags.
-
-If while running benchmarks your redis-server is killed every time the program runs the 4000 data-size benchmark, it might be because you don't have enough available storage on your machine.
-To solve this issue, you have two options -
-
-1. Allocate more storage to your'e machine. for me the case was allocating from 500 gb to 1000 gb.
-2. Go to benchmarks/install_and_test.sh and change the "dataSize="100 4000"" to a data-size that your machine can handle. try for example dataSize="100 1000".
-
-## Additional packages
-
-For python, run `sudo apt install python3.10-venv` (with the relevant python version), in order to be able to create a virtual environment.
-In order to run rust-analyzer through remote VScode, you need to run `sudo apt install pkg-config libssl-dev`.
-
 ## Recommended VSCode extensions
 
-### General
-
 [GitLens](https://marketplace.visualstudio.com/items?itemName=eamodio.gitlens)
-
-### Rust development
-
-[rust-analyzer](https://marketplace.visualstudio.com/items?itemName=rust-lang.rust-analyzer) - Rust language server.
-[CodeLLDB](https://marketplace.visualstudio.com/items?itemName=vadimcn.vscode-lldb) - Debugger.
-[Even Better TOML](https://marketplace.visualstudio.com/items?itemName=tamasfe.even-better-toml) - TOML language support.
-
-### C# development
-
-[C#](https://marketplace.visualstudio.com/items?itemName=ms-dotnettools.csharp) - lightweight language server and in-editor test runner.
-
-### TypeScript development
-
-[Prettier - Code formatter](https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode) -JavaScript / TypeScript formatter.
-[Jest Runner](https://marketplace.visualstudio.com/items?itemName=firsttris.vscode-jest-runner) - in-editor test runner.
-[Jest Test Explorer](https://marketplace.visualstudio.com/items?itemName=kavod-io.vscode-jest-test-adapter) - adapter to the VSCode testing UI.
-[ESLint](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint) - linter.
-
-### Python development
-
-[Python](https://marketplace.visualstudio.com/items?itemName=ms-python.python) - Language server
-[Black Formetter](https://marketplace.visualstudio.com/items?itemName=ms-python.black-formatter) - Formatter

babushka-core/README.md

@@ -0,0 +1,7 @@
+# Rust core
+
+## Recommended VSCode extensions
+
+[rust-analyzer](https://marketplace.visualstudio.com/items?itemName=rust-lang.rust-analyzer) - Rust language server.
+[CodeLLDB](https://marketplace.visualstudio.com/items?itemName=vadimcn.vscode-lldb) - Debugger.
+[Even Better TOML](https://marketplace.visualstudio.com/items?itemName=tamasfe.even-better-toml) - TOML language support.

benchmarks/README.md

@@ -0,0 +1,11 @@
+# Benchmarks
+
+[`install_and_test.sh`](./install_and_test.sh) is the benchmark script we use to check performance. run `install_and_test.sh -h` to get the full list of available flags.
+
+The results of the benchmark runs will be written into .csv files in the `./results` folder.
+
+If while running benchmarks your redis-server is killed every time the program runs the 4000 data-size benchmark, it might be because you don't have enough available storage on your machine.
+To solve this issue, you have two options -
+
+1. Allocate more storage to your'e machine. for me the case was allocating from 500 gb to 1000 gb.
+2. Go to benchmarks/install_and_test.sh and change the "dataSize="100 4000"" to a data-size that your machine can handle. try for example dataSize="100 1000".

csharp/README.md

@@ -0,0 +1,7 @@
+# C# wrapper
+
+The C# wrapper is currently not in a usable state.
+
+## Recommended VSCode extensions
+
+[C#](https://marketplace.visualstudio.com/items?itemName=ms-dotnettools.csharp) - lightweight language server and in-editor test runner.

node/README.md

@@ -1,23 +1,50 @@
-This package contains an internal package under the folder `rust-client`, which is just the Rust library and its auto-generated TS code. The external package, in this folder, contains all wrapping TS code and tests.
+# NodeJS wrapper
 
-### First time install
+This package contains an internal package under the folder `rust-client`, which is just the Rust library and its auto-generated TypeSCript code. The external package, in this folder, contains all wrapping TypeSCript code and tests.
+
+## Building
+
+### dependency installation
 
 Homebrew is the easiest way to install node, but you can choose any way you want to install it.
 Npm should come as part as node. IMPORTANT - right now we support only npm major version 8.
 The default install will be of version 9, so you'll need to downgrade - run `npm i -g npm@8`.
 
-### Build
+### Adding the babushka package to your package
+
+Before you add the babushka package to your application, make sure to perform the stages in "[Building the wrapper](#building-the-wrapper)".
+ATM Babushka isn't on npm, so you'll need to clone this repo to you device, and add the package using a folder path - `npm install <path to Babushka>/node`.
+
+### Building the wrapper
+
+Before you start, make sure that you have all dependencies mentioned [here](../README.md#development-pre-requirements), [rustup](../README.md#rustup), [node](../README.md#node-16-or-newer) installed, and all [submodules updated](../README.md#git-submodule).
+Enter this folder in your terminal.
+On the first install, run `npm i`, to install all dependencies. Also enter the `rust-client` folder, and run `npm i` there, too.
 
-Run `npm run build-internal` to build the internal package and generates TS code. `npm run build-external` builds the external package without rebuilding the internal package. Run `npm run build-protobuf` to generate node's protobuf files.
-Run `npm run build` runs a full build.
+make sure that all dependencies, including rustup, have been installed.
+Run `npm run build:release` runs a full build in release mode. This might take a while.
+
+For testing purposes, you can run `npm run build` runs a full, unoptimized build.
+If you've only made changes to the Rust code, run `npm run build-internal` to build the internal package and generates TypeSCript code.
+If you've only made changes to the TypeScript code, run `npm run build-external` to build the external package without rebuilding the internal package.
+Run `npm run build-protobuf` to generate node's protobuf files.
+
+Once building completed, you'll find the compiled JavaScript code in the ./build-ts folder,
 
 ### Testing
 
 Run `npm test` after building.
 
-### [Optional] Manually compile protobuf files 
+### [Optional] Manually compile protobuf files
 
 1. Run `npm install protobufjs-cli` to install the JS protobuf command line utility
 2. Generate static JS code `pbjs -t static-module -o ProtobufMessage.js ~/babushka/babushka-core/src/protobuf/*.proto`
-3. Generates TS definitions from the compiled JS file `pbts -o ProtobufMessage.d.ts ProtobufMessage.js`
+3. Generates TypeSCript definitions from the compiled JS file `pbts -o ProtobufMessage.d.ts ProtobufMessage.js`
 4. Due to a bug in the autogenerated files (see https://github.com/protobufjs/protobuf.js/issues/1063), manually replace `return this.encode(message, writer).ldelim();` to `return this.encode(message, writer && writer.len ? writer.fork() : writer).ldelim();` in the encodeDelimited functions of Request and Response under babushka/node/src/ProtobufMessage.js.
+
+## Recommended VSCode extensions
+
+[Prettier - Code formatter](https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode) -JavaScript / TypeScript formatter.
+[Jest Runner](https://marketplace.visualstudio.com/items?itemName=firsttris.vscode-jest-runner) - in-editor test runner.
+[Jest Test Explorer](https://marketplace.visualstudio.com/items?itemName=kavod-io.vscode-jest-test-adapter) - adapter to the VSCode testing UI.
+[ESLint](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint) - linter.

python/README.md

@@ -1,7 +1,11 @@
+# Python Wrapper
+
 Using [Pyo3](https://github.com/PyO3/pyo3) and [Maturin](https://github.com/PyO3/maturin).
 
 ### Create venv
 
+Install python-venv: run `sudo apt install python3.10-venv` (with the relevant python version), in order to be able to create a virtual environment.
+
 `cd python` cd into babushka/python folder
 `python -m venv .env` in order to create a new virtual env.
 
@@ -36,14 +40,14 @@ Autogenerate python's protobuf files with:
 
 Run `pytest --asyncio-mode=auto` from this folder, or from the `tests` folder. Make sure your shell uses your virtual environment.
 
-#### Running linters:
+#### Running linters
 
 Using VS code, install the following extensions:
 
--   Python
--   isort
--   Black Formatter
--   Flake8
+-   [Python](https://marketplace.visualstudio.com/items?itemName=ms-python.python)
+-   [isort](https://marketplace.visualstudio.com/items?itemName=ms-python.isort)
+-   [Black Formetter](https://marketplace.visualstudio.com/items?itemName=ms-python.black-formatter)
+-   [Flake8](https://marketplace.visualstudio.com/items?itemName=ms-python.flake8)
 
 Or, run in a terminal: