This repository hosts custom Greengrass components

Sample spring boot application for AWS elasticbeanstalk with fluentd and kinesis configurations

Real Time Data Ingestion, Change Data Capture, In Transit Data Manipulation using DynamoDB, DynamoDB Stream, Kinesis Stream, Kinesis Firehose, Lambda Resources

The purpose of this repository is to apply a data ingestion with Amazon Kinesis Firehose saving that data to S3 using the boto3. After that I use AWS Glue to catalog and Athena to query the data.

Serverless Data pipeline with Api Gateway, AWS lambda, Amazon Kinesis, S3 and Snoflake

IaC to simulate a microsservice hosted in AWS based on messages

ETL using application streaming and creating a Data Lake

A project using AWS to combine and transform streaming data for analytics.

Keywords: Python, AWS Kinesis, AWS Firehose, AWS Lambda, Snowflake, AWS S3, AWS Athena

Analyze real-time global market data using AWS Kinesis and Snowflake. We utilize CSV datasets extracted via API calls, stream them through Kinesis Firehose, and transform them with Snowflake. Our agile workflow ensures efficiency, providing a one stop comprehensive solution for real-time data insights.

Serverless big data streaming application using a fan-out architecture with Lambda and SQS to fetch data from Reddit forums, stream to Kinesis Firehose and store in an S3 data lake.

Receives data from firehose and puts it to ElasticSearch

Ship CloudWatch Logs to Firehose!

Cross account log exports of S3 access logs, CWLogs, Cloudtrail, GuardDuty and Ec2 custom log upload to CWLogs

Creates an EC2 webserver instance in a VPC with logs sent to Cloudwatch and to s3 for long term storage

Loan Data Feed Application

The simple solution that resolves problem adding new partitions created by Kinesis Firehose into Amazon Athena

This workshop is to build a serverless data lake architecture using Amazon Kinesis Firehose for streaming data ingestion, AWS Glue for Data Integration (ETL, Catalogue Management), Amazon S3 for data lake storage, Amazon Athena for SQL big data analytics.

A demo of DynamoDB CDC into data lake with AWS CDK v2

An end-to-end AWS data engineering pipeline that orchestrates data ingestion, processing, and storage using Kinesis, S3, Redshift, Lambda, DynamoDB