Dynamic-Web-Scrapper-and-Analyst

Web Scraping and AI Text Generation

This project combines web scraping and AI text generation techniques to extract data required like from Movie Webiste movie titles, years, ratings, and other information like analysis using Puppeteer and then generating additional text with OpenAI's GPT-3.5 Turbo.

Main Points :-

:- To run it seperately , get the code from the requests file (from the get request) as this is the demo verison of API.
:- Adjust the tokens as per needs as OPEN API if free version is being used then it may gie error for Run time limit.

Table of Contents

• Introduction
• Technologies Used
• Getting Started
• Usage
• Examples
• Contributing
• License

Introduction

This project demonstrates how to scrape information from a web page and process it with artificial intelligence. In this specific case, we're scraping movie data from Rotten Tomatoes and using GPT-3.5 Turbo to generate additional text based on the scraped content.

Technologies Used

• Puppeteer: A Node.js library for controlling headless browsers.
• OpenAI GPT-3.5 Turbo: A powerful text generation model by OpenAI.
• JavaScript: The primary programming language used.
• Node.js: The runtime environment for executing JavaScript on the server.
• Other dependencies (list them if you have additional libraries)

Getting Started

To run this project locally, follow these steps:

1. Clone the repository: git clone https://github.com/yourusername/your-repo.git
2. Install the necessary dependencies: npm install
3. Set up your OpenAI API key in the code: Replace your-api-key with your actual API key.

Usage

To use this project, follow these steps:

1. Run the script: node your-script.js
2. The script will scrape movie data and generate additional text.
3. Customize the prompt template in your code for different text generation tasks.

Examples

Here are some examples of how to use this project effectively:

• [Add examples of different use cases or scenarios where this project can be applied.]

Contributing

Contributions are welcome! If you'd like to contribute to this project, please follow these guidelines:

1. Fork the repository.
2. Create a new branch for your feature or bug fix: git checkout -b feature/your-feature-name
3. Make your changes and commit them with descriptive commit messages.
4. Push your changes to your fork: git push origin feature/your-feature-name
5. Create a pull request to the main repository.

---