spark adls read write

Tutorial on setup of online and offline feature stores in Azure Machine Learning (Azure ML), using command line interface (Az CLI)

Workshop to build scalable architectures with Azure Data Explorer (ADX)

Terraform module for terraform-azure-adls-gen1

An end-to-end ETL pipeline that extracts data from an Azure SQL Server database, transforms the data using Databricks, and loads the transformed dataset into Azure Data Lake Storage (ADLS).

Upload data onto different cloud storages by HDFS client

End to end solution for writing USQL data into parquet and setup Presto to query the data

Kafka Connect ADLS - a Kafka Sink connector for ADLS Gen1

Implementing Change Data Capture for Seamless Fintech Data Migration

ADLSGen2-Databricks-Connectivity

Internship on Data Engineering where below topics are applied skills that are used to complete the given tasks through out 8 weeks including the project.

Terraform module for terraform-azure-adls-gen2

HDInsight provider for Airflow

Activity Assistant provides a platform for logging, evaluating and predicting Activities of Daily Living for Home Assistant.

File Source to read data from distributed file system

Fluentd output plugin for Azure Datalake Storage Gen2 (append support)

Contains data preprocessing and visualization methods for ADL datasets.

Real-time ADLs and Fall Detection implement TensorFlow