spark-kafka-integration

Here are 12 public repositories matching this topic...

spider-123-eng / Spark

Apache Spark is a fast, in-memory data processing engine with elegant and expressive development API's to allow data workers to efficiently execute streaming, machine learning or SQL workloads that require fast iterative access to datasets.This project will have sample programs for Spark in Scala language .

streaming consumer parquet kafka-producer spark-sql spark-kafka-integration spark-streaming-data spark-transformations spark-to-cassandra-connection spark-dataframes spark-joins spark-hive-context spark-jdbc-connection spark-with-mangodb spark-aggregations-using-dataframe spark-use-cases cassandra-installation spark-datadog spark-mangodb spark-catalog-api

Updated Nov 16, 2022
Scala

zekeriyyaa / PySpark-Structured-Streaming-ROS-Kafka-ApacheSpark-Cassandra

Star

A structured streaming was applied to the robot data from ROS-Gazebo simulation environment using Apache Spark. Data is collected in Kafka, analyzed by Apache Spark and stored in Cassandra.

python apache-spark cqlsh python3 ros pyspark spark-streaming kafka-consumer data-analysis apache-kafka kafka-producer apache-cassandra structured-streaming spark-sql spark-kafka-integration spark-cassandra-connector spark-cassandra ros-noetic spark-kafka-connector

Updated Feb 6, 2022
Python

akashsethi24 / Spark-Kafka-Example

Star

Example for Data Reading from and Writing to from Kafka Topic using Apache Spark DataFrame and DataSet

kafka apache-spark spark-streaming kafka-streams spark-kafka-integration custom-kafka-serializer custom-kafka-deseriazer kafka-decoder kafka-encoder

Updated Oct 13, 2017
Scala

xuanbo / spark-programming

Star

spark代码

spark spark-streaming spark-sql spark-kafka-integration

Updated Jun 21, 2022
Scala

mayankrawat / CSVJoin

Star

Use this project to join data from multiple csv files. Currently in this project we support one to one and one to many join. Along with this you can find how to use kafka producer efficiently with spark.