usage.md

Betflow

Contents of Repository

1. Direction
2. Infra Setup
3. Batch Pipelines
4. Demo
5. Running pipelines
6. Dashboard

1. Directions to use

1. Clone repository git clone https://github.com/sanchitvj/sports_betting_analytics_engine.git
2. Change directory cd sports_betting_analytics_engine
3. Install package betflow using pip install .
   If you want to install in editable mode pip install -e .
4. Make an environment file named my.env and add keys for the APIs you want to try

ODDS_API_KEY=
OPEN_WEATHER_API_KEY=
GLUE_ROLE=
GITHUB_TOKEN=
GNEWS_API_KEY=

2. Infrastructure Setup

Check below links for there setups
Apache Airflow
Apache Druid
Apache Kafka and S3-Connector
dbt
Grafana
Snowflake

3. Batch Pipelines

Detailed explanation of pipelines and lineage:
Airflow
dbt

4. Demo screenshots

Demo

5. Running real-time pipelines

• cd src/pipeline_mains
• Games: python games_pipeline
• Odds: python odds_pipeline
• Weather: python weather_pipeline

6. Dashboard

Some of the queries used for visualization in Grafana are here.