The Main Problemset of BUET CSE FEST 2024 Hackathon is Given Here: https://docs.google.com/document/d/1ZUuS373rbKFZw3HPlVrvPgzdF4Xw7ktsbiI5TjyNE2s/mobilebasic
The Problems in our understanding:
- Solve Time-consuming research across multiple platforms
- Difficulty in coordinating transportation and accommodation
- Solve Complex budget management
- Show efficient Maps and Smart Weather Notification
- Solve Manual organization of travel memories
- Problem of Uploading and Sharing Trip Images with Trip mates(Haha RIP Whatsapp)
- Searching Images Fast and Quick using Semantic Search
- Generate Automated Blogs and Vlogs from Trip Information
EasyTrip addresses these challenges through an integrated platform that leverages langchain of ai agents, real-time data, and automation to provide a seamless travel planning experience. By offering smart itinerary generation, interactive map integration, weather intelligence, and automated content generation, EasyTrip simplifies the trip planning process and enhances the overall travel experience for users.
- Set up API keys in
python-server/.env. - Navigate to the
python-serverdirectory:cd python-server - Install the required dependencies:
pip install -r requirements.txt
- Start the Python backend:
python trip_api.py
- Set up
.envwith all credentials and database URLs. - Install the required dependencies:
npm i --force
- Apply database migrations:
npx prisma migrate dev
- Generate the Prisma client:
npx prisma generate
- Start the Next.js development server:
npm run dev
Build and start the Docker containers:
sh docker-compose build docker-compose up
- Natural language input for Trip generation
- Traditional and AI-powered itinerary generation
- Detailed day-wise itinerary with activities with proper Validation
- Google Search API Integration for fetching real-time data
- Customizable preferences for:
- Budget range
- Travel style
- Duration
- Accommodation preferences
- Cost breakdown and estimation
- Real-time availability checking
- Accommodation and transportation booking
- Multi-destination trip planning with checkpoints
- Add friends to the Trip
Workflow:
- Dynamic route visualization with multiple stops and modes of transport
- Points of interest mapping
- Real-time location updates
- Interactive markers with detailed information
- Distance and travel time calculations
Workflow:
- Real-time weather updates
- Weather forecast for next 7 days and also for the trip duration
- Weather-based itinerary adjustments
- We run Corn Job which sends weather updates to all active TripPlans
- Severe weather alerts
- Historical weather pattern analysis
- Weather-optimized activity suggestions
Workflow:
- User Action: The user can request a trip blog or manually input blog content.
- Data Retrieval: The system fetches trip itinerary and user profile information.
- Event and Memory Analysis: It identifies notable events/locations and gathers user memories and preferences.
- Semantic Photo Search: Photos are searched and organized based on trip relevance.
- Draft Blog Creation: A personalized blog draft is generated using the query, photos, and trip data.
- User Review and Actions: The user can edit, finalize, or discard the blog draft.
- Saving or Deletion: Finalized blogs are stored in the database, while discarded drafts are removed from the system.
- User Notification: The user receives a notification upon blog publication.
Workflow:
- Image upload and organization
- Textual search for images embeddings of Dimention 1536 (Using OpenAi text Embedding)
- Image tagging and categorization
- Vector Similarity Search using lencedb
Workflow:
- User Query Input: : The user provides a query specifying the topic for the video vlog.
- Collect Trip Data: : The system gathers relevant images and text contexts from the trip based on the query.
- Generate Subtitle: : A subtitle is created using an expert subtitle writing system.
- Combine Images and Subtitle: : The collected images and generated subtitle are combined to structure the vlog content.
- Generate Audio for Subtitle: : Audio narration is generated based on the subtitle to complement the visual content.
- Assemble the Complete Vlog: : The system combines the images, subtitles, and audio to generate the final video vlog.
- User Review and Modification: : The user can edit or modify the vlog before finalizing it.
- Post to Platform: : The finalized vlog is published on the platform.
- Notify TripMates: : All tripmates receive notifications about the new vlog post.
Workflow:
- Chat Interface: : The user opens the chat interface to interact with the Trip Companion AI, similar to ChatGPT.
- User Query Submission: : The user submits a trip-related query or asks a question about their travel plans.
- AI Knowledge Base Access: : The AI accesses the trip’s knowledge base, which includes itinerary details, locations, events, and user-specific preferences.
- Query Analysis and Data Retrieval: : The AI analyzes the user’s query and retrieves relevant information from the knowledge base.
- Generate and Provide Response: : Based on the gathered trip data, the AI generates an accurate response or suggestion.
- Response Display: : The response is displayed to the user within the chat interface.
- User Follow-Up Actions: : The user can ask additional queries or end the chat session.
-
Frontend:
- Next.js 14 (React Framework)
- Tailwind CSS (Styling)
- Shadcn/UI (Component Library)
- TypeScript
-
Backend:
- Python (FastAPI)
- LangChain
- PostgreSQL Database + LanceDB
- Firebase Image Storage
- Next Auth (Authentication)
-
AI/ML Services:
- OpenAI API
- Custom LangChain Agents
-
External APIs:
- Google Maps Platform
- Weather API (http://api.weatherapi.com/v1/)
- Cloud Storage (AWS RDS)
- Google Search API (https://serper.dev/)
- The Next.js frontend acts as the user interface, sending requests to the API Gateway.
- The API Gateway directs requests to backend services for authentication, itineraries, and weather.
- The Authentication Service secures user sessions, while the User Database holds user information.
- The Itinerary Service uses the OpenAI API for trip planning and the Google Maps API for route visualization.
- Firebase Image Storage handles user-uploaded images, with all data stored in PostgreSQL + LanceDB for easy retrieval.
API Documentation for EasyTrip Platform can be seen after starting python-server on the following route:
1. Natural Language Based AI Trip Generation:
ENDPOINT: http://localhost:3000/api/tripPlan/extract?<USER_PROMPT>
Content-Type: application/json
Method: GET
Example:
GET http://localhost:3000/api/tripPlan/extract?text=%22journey+from+dhaka+to+sylhet+with+3+friends+4+days+budget+5k%22
Response:
{
"output": {
"origin": "Dhaka",
"destination": "Sylhet",
"days": "4",
"budget": "5000",
"people": "4",
"preferences": "",
"tripType": "oneWay",
"journeyDate": "today",
"travelClass": "economy"
},
"metadata": {
"run_id": "3e2d0a89-be92-45ef-8e10-1d64f7bd24ec",
"feedback_tokens": []
}
}
2. Manually generate an itinerary based on user preferences:
ENDPOINT: http://localhost:3000/api/tripPlan?<QUERY_PARAMS>
Content-Type: application/json
Method: GET
Example:
GET http://localhost:3000/api/tripPlan?origin=%22dhaka%22&destination=%22sylhet%22&days=%223%22&budget=%222000%22&people=%224%22&preferences=%22hill%22&tripType=%22oneWay%22&journeyDate=%2224/10/2024%22&travelClass=%22economy%22
Response:
{
"output": {
"trip_name": "Dhaka to Sylhet 3 days trip with 4 people to enjoy the hill",
"origin": "Dhaka",
"destination": "Sylhet",
"days": "3",
"budget": {
"total": "2000",
"breakdown": {
"transportation": "800",
"food": "600",
"accommodation": "500",
"miscellaneous": "100"
}
},
"people": "4",
"preferences": "hill",
"tripType": "oneWay",
"journeyDate": "24/10/2024",
"travelClass": "economy",
"checkpoints": [
{
"origin": {
"location": "Dhaka",
"latitude": "23.8103",
"longitude": "90.4125"
},
"destination": {
"location": "Sylhet",
"latitude": "24.8949",
"longitude": "91.8687"
},
"logistics": {
"departure_time": "06:00 AM",
"arrival_time": "12:00 PM",
"tips": "Take an early morning bus from Dhaka to Sylhet to enjoy the scenic beauty along the way."
}
}
],
"food": {
"1": {
"breakfast": {
"title": "Panshi Restaurant",
"address": "Jallarpar Rd, Sylhet 3100",
"latitude": 24.895068799999997,
"longitude": 91.8674443,
"rating": 4.2,
"ratingCount": 18000,
"category": "Bangladeshi restaurant",
"phoneNumber": "01761-152939",
"cid": "4184260984599101480",
"cost": "150"
},
"launch": {
"title": "Pach Bhai Restaurant",
"address": "Jallarpar Rd, Sylhet 3100",
"latitude": 24.8946981,
"longitude": 91.8664029,
"rating": 4.3,
"ratingCount": 16000,
"category": "Bangladeshi restaurant",
"phoneNumber": "01710-459607",
"cid": "1251724275242512479",
"cost": "200"
},
"dinner": {
"title": "The Mad Grill",
"address": "Nayasarak Point, Manik Pir Road, 3100",
"latitude": 24.8995748,
"longitude": 91.87515789999999,
"rating": 4.3,
"ratingCount": 2300,
"category": "Restaurant",
"phoneNumber": "01954-556677",
"website": "https://www.facebook.com/themadgrill/",
"cid": "9696671651361504064",
"cost": "250"
}
}
},
"accommodation": {
"1": {
"title": "The Grand Hotel",
"address": "4th Floor, H. S. Tower, HS Tower 3rd Floor Waves -1 East, Waves-1 Dargah Gate, Sylhet 3100",
"latitude": 24.901723,
"longitude": 91.86977929999999,
"rating": 4,
"ratingCount": 564,
"category": "Hotel",
"phoneNumber": "01970-793366",
"cid": "16335710689796874260"
},
"2": {
"title": "Hotel Noorjahan Grand",
"address": "Waves 1 Dargah Gate, Sylhet 3100",
"latitude": 24.901979599999997,
"longitude": 91.8696968,
"rating": 4.2,
"ratingCount": 2900,
"category": "Hotel",
"phoneNumber": "01930-111666",
"website": "http://www.noorjahangrand.com/",
"cid": "15253580246980481310"
}
}
},
"metadata": {
"run_id": "6fff0355-b8e0-47c9-9776-40664d30d169",
"feedback_tokens": []
}
}PUT /api/itinerary/:id
Content-Type: application/json
{
"updates": {
"accommodation": "luxury",
"activities": ["scuba-diving"]
}
}GET http://api.weatherapi.com/v1/<QUERY_PARAMS>GET http://maps.googleapis.com/maps/api/directions/json?origin=Dhaka&destination=Sylhet&key=YOUR_API_KEY
POST /api/image
Content-Type: multipart/form-data
{
"image": "file"
}GET /api/image/search?query=beacheasytrip/
├── app/ # Next.js app directory
│ ├── +---(auth)
| |── +---signin
| | page.tsx
| |
| |── +---trip
| | | layout.tsx
| | | page.tsx
| | |
| | +---ai-bot
| | | page.tsx
| | |
| | +---blogs
| | | generate.tsx
| | | page.tsx
| | |
| | +---maps
| | | page.tsx
| | |
| | +---media
| | | firebase.ts
| | | page.tsx
| | | uploads.tsx
| | |
| | +---preview
| | | page-old.tsx
| | | page.tsx
| | |
| | +---settings
| | | page.tsx
| | |
| | \---vlogs
| | page.tsx
| | text_to_speech.ts
| |
| -+---api
| | +---add-member
| | | route.tsx
| | |
| | +---auth
| | | \---[...nextauth]
| | | route.ts
| | |
| | +---blog
| | | route.ts
| | |
| | +---chat
| | | route.ts
| | |
| | +---image
| | | route.ts
| | |
| | +---test
| | | route.ts
| | |
| | +---trip
| | | route.tsx
| | |
| | +---tripPlan
| | | | route.tsx
| | | |
| | | \---extract
| | | route.tsx
| | |
| | +---trpc
| | | | trpc-router.ts
| | | |
| | | \---[trpc]
| | | route.ts
| | |
| | +---uploadthing
| | | core.ts
| | | route.ts
| | |
| | +---vlog
| | | route.ts
| | |
| | \---weather
| | route.tsx
├── components/ # React components
│ ├── ui/ # UI components
│ └── features/ # Feature components
├── lib/ # Shared utilities
│ ├── utils/ # Helper functions
├── python-server/ # FastAPI backend
│ ├── trips/
│ ├── trip_api # handelar functions
└── public/ # Static assets