Repositories list

• data-consumer-pipeline

  Public
  Pipeline for processing and consuming streaming data from Pub/Sub, integrating with Dataflow for real-time data processing

  pythondataflowapache-beam+ 1data-pipeline

  Python

  •

  MIT License

  •0•0•1•1•Updated Nov 18, 2024Nov 18, 2024

• data-producer-api

  Public
  FastAPI application for sending data to Pub/Sub, used for load testing and triggering pipelines

  cloudgcpdata-engineering+ 2api-restfastapi

  Python

  •

  MIT License

  •0•1•0•4•Updated Nov 18, 2024Nov 18, 2024

• api-to-dataframe

  Public
  Python library that simplifies obtaining data from API endpoints by converting them directly into Pandas DataFrames. This library offers robust features, including retry strategies for failed requests.

  pythonlibrarydata-engineering+ 3data-analyticsdata-analysispypi-packages

  Python

  •

  MIT License

  •0•0•0•1•Updated Nov 18, 2024Nov 18, 2024

• data-consumer-api

  Public
  ETL Process for Currency Quotes Data" project is a complete solution dedicated to extracting, transforming and loading (ETL) currency quote data. This project uses several advanced techniques and architectures to ensure the efficiency and robustness of the ETL process.

  pythondata-visualizationdata-engineering+ 5business-intelligencedata-analyticsdata-analysisdata-pipelineetl-pipeline

  Python

  •

  MIT License

  •1•2•1•4•Updated Nov 18, 2024Nov 18, 2024

• currency-quote

  Public
  Complete solution for extracting currency pair quotes data with comprehensive testing, parameter validation, flexible configuration management, Hexagonal Architecture, CI/CD pipelines, code quality tools, and detailed documentation.

  pythonlibrarydata-engineering+ 3data-analyticsdata-analysispypi-packages

  Python

  •

  MIT License

  •0•0•0•1•Updated Nov 18, 2024Nov 18, 2024

• .github

  Public
  0•0•0•0•Updated Nov 1, 2024Nov 1, 2024