parma-mining-clearbit

ParmaAI mining module for the clearbit CRM.

Getting Started

The following steps will get you started with the project.

1. Pre-requisites: to be able to contribute to JST in this repository, make sure to comply with the following prerequisites.

   • Configure GitHub via an ssh key. Key based authenticated is highly encouraged. See GitHub Docs for more information.
   • Please make sure to have an GPG key configured for GitHub. See GitHub Docs for more information.
   • Install micromamba, a conda environment management package manager, as described here. Alternatively conda or mamba installations should also work, but are highly discouraged because of their slow performance.

2. Clone the repository

   git@github.com:la-famiglia-jst2324/parma-mining-clearbit.git

3. Precommit & environment setup:

   make install  # execute the last 2 steps manually!

4. Start the api server:

   make dev

   Open http://localhost:8000 with your browser to see the result.

   FastApi will provide you with an interactive documentation of the api. You can also use the swagger ui at http://localhost:8000/docs or the redoc ui at http://localhost:8000/redoc.

5. Optional: Running the pre-commit pipeline manually

   pre-commit run --all

6. Test your code:

   make test

PR workflow

1. Create a new branch linear.app offers a button to copy branch names from tickets. In case there is no ticket, please use feel free to use an arbitrary name or create a ticket. GitHub CI doesn't care about the branch name, only the PR title matters.

   # format: e.g. robinholzingr/meta-1-create-archtecture-drafts-diagrams-list-of-key-priorities
   git checkout -b <branch-name>

2. Open a PR and use a conventional commit PR title.

3. Wait for CI pipeline to pass and if you are happy with your changes request a review.

4. Merge the PR (using the "Squash and merge" option) and delete the branch. Pay attention to include co-authors if anyone else contributed to the PR.

5. If you want to release a new version to production, create a new release on GitHub. The release version will be automatically derived from the PR titles (breaking changes yield new major versions, new features yield new minor versions).

Directory structure

.
├── parma_mining.clearbit: Main sourcing code
│   └── api: FastAPI REST API
├─ tests: Tests for mining module
├── Makefile: Recipes for easy simplified setup and local development
├── README.md
├── docker-compose.yml: Docker compose file for local database
├── environment.yml: conda environment file
└── pyproject.toml: Python project configuration file

Tech Stack

Core libraries that this project uses:

• FastAPI: FastAPI is a modern, fast (high-performance), web framework for building APIs with Python 3.6+ based on standard Python type hints.
• Pydantic: Data validation and settings management using python type annotations.
• Typer: Typer is a library for building CLI applications that users will love using and developers will love creating.
• Polars: Polars is a blazingly fast data processing library written in Rust. It has a DataFrame API that is similar to Pandas and a Series API that is similar to NumPy.
• Pytest: The pytest framework makes it easy to write small tests, yet scales to support complex functional testing for applications and libraries.

Deployment

No deployment pipeline has been set up yet.

Currently we are considering several backend frameworks like Firebase, Supabase or AWS Amplify.

Disclaimer

In case there are any issues with the initial setup or important architectural decisions/integrations missing, please contact the meta team or @robinholzi directly.