Large Graph Processing with Apache Spark

The repo contains graph algorithms based on Pregel implemented from scratch.

Quick start

The easiest way to start is to use one of the cloud providers (free):

• . When loaded go to: src -> pregel.ipynb;
• . When loaded go to work -> pregel.ipynb.

Local setup

In order to setup the local environment one needs to install Docker and Git.

When the needed software installed, please execute:

1. clone the repo: git clone https://github.com/neshkeev/spark-graphs-demo.git
2. enter the directory: cd spark-graphs-demo
3. execute: docker compose up
4. open Jupyter Lab in the web browser: http://localhost:8888/lab;
5. In the sidebar with files go to: work -> pregel.ipynb.