-
python manage.py runserver
-
redis-server
-
celery -A tech_trends beat --loglevel=info
-
celery -A tech_trends worker --pool=solo -l info
This project is a Django-based API for scraping and updating tech news trends. It uses Celery for asynchronous task management, Redis as a message broker, and a custom scraper to fetch the latest tech news.
- Tech News Scraping: Automatically fetches and updates the latest tech news trends.
- Celery Integration: Handles asynchronous tasks for updating trends every 5 minutes.
- Redis: Used as a message broker for Celery tasks.
- Django REST Framework: Provides endpoints to access the latest tech trends.
- Python
- Redis
- Django
- Clone the repository:
git clone https://github.com/kushal1o1/tech_trends.git
cd tech_trends
- Create a virtual environment and activate it:
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
- Install dependencies:
pip install -r requirements.txt
-
Setup Redis:
- Ensure Redis is installed and running on localhost:6379.
- If Redis is not installed, follow the installation guide here.
-
Configure Django settings:
- In tech_trends/settings.py, set your database configurations and any other necessary settings.
- Ensure the following Celery settings are configured:
CELERY_BROKER_URL = 'redis://localhost:6379/0'
CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'
CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_TIMEZONE = 'UTC'
- Run migrations:
python manage.py migrate
- Start the development server:
python manage.py runserver
-
Run Celery worker and beat:
Open two separate terminal windows and run the following commands:
Celery Worker:
celery -A tech_trends worker --pool=solo -l info
Celery Beat:
celery -A tech_trends beat --loglevel=info
- The API will automatically fetch and update the latest tech trends every 5 minutes.
- Access the latest trends through the provided endpoints.
- GET /api/trends/ - Fetch the latest tech trends.
Feel free to open issues or submit pull requests if you'd like to contribute to this project.
This project is licensed under the MIT License.
Author: Kushal Baral