This post is focused on how to setup Apache Airflow 2 using DockerOperator with node.js and Gitlab container registry on Ubuntu 20
-
Updated
Jul 16, 2021 - Python
This post is focused on how to setup Apache Airflow 2 using DockerOperator with node.js and Gitlab container registry on Ubuntu 20
This Python-based project extracts data from Wikipedia using Apache Airflow, cleans it and pushes it Azure Data Lake for processing and further processing and visualization is done on Azure Data Factory, Azure Synapse and Tableau.
Utility Scripts and Functions for Apache Airflow
This project involves development of a ETL data pipeline that allows streaming millions of Amharic and Swahili speech audio files and speakers providing transcription texts for data collection in a web platforms.
A small dag workflow I created using Apache Airflow
Real-time ETL (Extract, Transform, Load) data pipeline to process insurance claims data with Snowflake, Apache Airflow, AWS S3, EC2, python pandas and creating a real time data visualization dashboard using Tableau.
WeatherFlow ETL is an automated pipeline that extracts weather data from public APIs, transforms it by cleaning and standardizing the information, and loads it into a PostgreSQL database for analysis.
Add a description, image, and links to the apacheairflow topic page so that developers can more easily learn about it.
To associate your repository with the apacheairflow topic, visit your repo's landing page and select "manage topics."