Skip to content

Latest commit

 

History

History
9 lines (8 loc) · 487 Bytes

README.md

File metadata and controls

9 lines (8 loc) · 487 Bytes

BasicRAGPipeline

Tried all the data ingestion techniques.

Learned to ingest text file using textloader of langchain.

Ingested data from a webpage after scraping it through beautifulsoup package in python.

Also, ingested data from pdfs using PyPDFloader package

Split the data into chunks to feed to the model.

Used OpenAI embeddings for vector embeddings.

Used chroma DB as well as FAISS (both) to store those vectors and hit them to retrieve the results.