padh.ai - now, you can “talk” to videos.

Table of Contents

• Introduction
• Features
• ScreenShots
• How It Works
• Tech Used

Introduction

padh.ai is An edtech app with courses, videos, chat, and FAQ.

What padh.ai aims at solving:

• Limited 1-on-1 interaction in online lectures
• Students getting distracted while looking up topics they need help with.
• Making doubt-solving more convenient and fast, feeding curiosity instantly.

Thus, enhancing overall academic performance and confidence of students

Features

• Browse and watch video courses
• Ask questions related to the video content or slides content using a chat interface
• View relevant materials like articles for each video

Screenshots

SignUp Screen	Login Screen
Dashboard 1	Dashboard 2
Topic Review	AV Summary Video Player
What is SOAP? 1	What is SOAP? 2
Context Aware LLM Answer	Context
Who is the Curator of Slides ?	Server Side Logs
More RAG Testing 1	More RAG Testing 2

How It Works!

padh.AI is built using Flutter for the frontend, FastAPI for the backend, OpenAI's GPT-3.5 turbo for querying, FAISS (Facebook AI Similarity Search) library in python for RAG (Retrieval Augmented Generation) and Firebase for authentication. Here's how it works:

1. User Authentication:

   • Users can sign up or log in using their email and password via Firebase authentication, or can utilize Google OAuth to login/sign in.

2. Courses Dashboard

   • Upon logging in, users are presented with a scrollable carousel of different courses and different Units under each course: A prospective redesign of the courses dashboard of the PESU App.
   • The User can select the topic that he wants to study. Each topic has a summary video along with relevant course content/slides.

3. Context-Aware Chatbot

   • The chatbot utilizes video and slide transcriptions stored in a vector database, providing accurate and context-specific answers to student queries.
   • A separate backend service converts each topic's summary video (audio to text using deepgram), and slides into vector embeddings. It stores them in a .faiss file so the chatbot is context-aware of that specific topic.

4. Advanced Query Processing

   • Using OpenAI’s GPT-3.5 turbo, student questions are matched with relevant content through FAISS, minimizing errors and enhancing the learning experience.

Tech-Used

• Frontend: Flutter - An open source cross-platform UI framework, using which the beautiful screens of bh.AI were built
• Backend: FastAPI - Backend for the app, uses Python, performs RAG embeddings on content as well as queries the LLM using system and user prompts
• langchain: Framework to integrate LLMs customized for different use Cases
• FAISS(Facebook AI Similarity Search): A library for efficient similarity search and clustering of dense vectors.
• OpenAI Embeddings: Used to created an information-dense representation of the semantic meaning of a piece of text.
• deepgram: Audio to Text ML model
• AI Querying: GPT 3.5 - The most popular LLM today.
• Authentication: Firebase - Backend as a Service software used to manage user authentication.
• Ngrok: Helps developers expose their local development server to the internet.