ellucidata

This project contains an efficient webscraping tool (Selenium) and data visualization dashboards.

Find out how food's price changes over time with our data visualization tool. Powered by a webscraper tool that daily collects up-to-date data from supermarkets's websites!

ellucidata.mp4

📖How does it work?

We developed an algorithm capable of scraping a website's interface, collecting data of a list of products in many supermarkets and send to a cloud MongoDB. Also, some dashboards were developed (using HMTL, CSS and Javascript) to allow analysis on products' price changes over time.

🧰Languages and Tools

• Python
• MongoDB
• Selenium
• JavaScript
• NodeJS

⁉️Why?

Our goal is to collect data from online supermarkets' products in order to analyse price changes during some period of time.

---

How can I run this locally?

💻 Prerequisites

Scraper

• Python3
• Chrome Driver (make sure that the driver's version matches Google Chrome's version in your computer)

Data Visualization

• NodeJS
• NPM

☁️ Clone this repository

https://github.com/e-llo/ellucidata.git

🌿 Install Dependencies

On the command line (Windows OS):

Scraper

python -m pip install -r requirements.txt

Visualization

npm install
npm run env

🏃‍♀️ Run

On the command line:

Scraper

python insert_data.py

Visualization

npm start

After the application is running, access http://localhost:1337/ from your browser

---

👩‍💻👨‍💻👨‍💻 Contributors

Julia Rolemberg

Pedro Heck

Raphael Menezes de Jesus