This repo contains implementations of PySpark for real-world use cases for batch data processing, streaming data processing sourced from Kafka, sockets, etc., spark optimizations, business specific bigdata processing scenario solutions, and machine learning use cases.

This is technically a RESTful API, but using PySpark module instead of the restful module! In this case, this is a template using PySpark for website development!

🐍💥Python and Spark for Big Data

This is a template API via PySpark!

Final submission. Topic: Apache Spark's Pyspark API

This is a template API via PySpark!

Designing and the implementation of different Spark applications to accomplish different jobs used to analyze a dataset on Covid-19 disease created by Our World In Data.

PySpark examples

PySpark from LinkedIn Learning: https://www.linkedin.com/learning/apache-pyspark-by-example/apache-pyspark

PySpark functions and utilities with examples. Assists ETL process of data modeling

An introductory notebook exploring the functionalities of Pyspark

Explains the implementation of spark concepts using pyspark API from jupyter notebook

Python scripts utilizing the PySpark API to convert a huge data set (about 3.5 GB) of flight data into various data storage formats such as CSV, JSON, Sequence file system