build: Add Continuous Deployment & terraform

[egekocabas]

Motivation

We need to deploy our data mining modules to the GCP.

Changes

• terraform files are added
• deploy.yml pipeline created
• Dockerfile is created
• release.yml pipeline deleted
• GITHUB_TOKEN is used inside pipeline and terraform

Checklist

• added myself as assignee
• correct reviewers
• descriptive PR title using conventional commits.
• description explains the motivation and details of the changes
• tests cover my changes
• documentation is updated
• CI is green
• breaking changes are discussed with the team and documented in the PR title ! (e.g. feat!: Update endpoint)

[linear]

META-13 Sourcing deployment

Currently blocked by Sourcing Team and their respective design decisions.

Firebase is blocked by #Meta-12

Please stay in touch with them and once they have agreed on sth. you can start deploying the respective services.

Expectations:

• Deployments to Firebase (cloud functions) / GCP (app service, …) / Apify depending on technical feasibility and desires of sourcing team
• respective CI pipelines for production, staging (and a local dev environment).
  • deploy on release to prod
  • deploy on merge to main to staging
  • local setup + docs
• Documentation! (preferably in READMEs, mermaid can help to visualise dependencies, workflows etc.)
• If feasible within this sprint assist sourcing team with local setup

[github-actions]

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines	Covered	Coverage	Threshold	Status
67	55	82%	0%	🟢

New Files

No new covered files...

Modified Files

No covered modified files...

updated for commit: 8bb42cf by action🐍

[egekocabas]

@robinholzi
Also in the analytics repository there are lock files under prod, staging and state folders. Do I need to create them? Because i searched a little bit and found out that they are created after terraform init or deploy, i am not really sure what to do about it actually.

Finally the last thing is that i need to add secret environment variables into this and all other mining repositories, how can i find them? I also have a comment about this topic in here.

[robinholzi]

Generally looks very good to me, the CI/CD worksflows (.github/workflows) are missing, right? Otherwise we won't ever use the terraform code :)

[robinholzi]

@egekocabas Let me know once this is ready for review (just transition the PR to review mode)

[robinholzi]

Looks super solid, thanks for taking the time to implement (and understand my terraform setup)🚀

Just one thing to check before merging!

[robinholzi]

Thanks for all the effort required for deploying this👍 I hope it was at least rewarding in terms of learning experience :)