Building a YouTube Scraper REST API with Puppeteer by following the TDD approach

Scrapes YouTube video data with ID, and returns title, description, channel, views and gameName (if it's a game video)

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.

Prerequisites

• Installed local Node.js environment
• Package manager to install packages. Yarn or npm

Clone the repo in your environment

git clone https://github.com/gnurgeldiyev/youtube-video-scraper.git

Installing

# move into project folder
cd youtube-video-scraper

# install the dependencies
yarn install

# rename the .env.sample
mv .env.sample .env

# add your variables
nano .env

Running the server

yarn start

Example

# GET - /{videoId}
curl localhost:3000/h8OX0FNWANM

Response

{
  "status": true,
  "title": "Recognizing Ignaz Semmelweis and Handwashing",
  "description": "Today’s Doodle follows the official guidelines ...",
  "channel": "GoogleDoodles",
  "views": "56188961"
}

Running the tests

yarn test

Built With

• Express - The web framework used
• Puppeteer - For web scraping
• Jest - For testing
• SuperTest - For testing HTTP API

License

This project is licensed under the MIT License - see the LICENSE.md file for details