Popular repositories Loading

1. mmore mmore Public

   Massive Multimodal Open RAG & Extraction A scalable multimodal pipeline for processing, indexing, and querying multimodal documents Ever needed to take 8000 PDFs, 2000 videos, and 500 spreadsheets …

   Python 20 4

2. MoE MoE Public

   some mixture of experts architecture implementations

   Python 12 2

3. nanotron nanotron Public

   Forked from huggingface/nanotron

   Minimalistic large language model 3D-parallelism training

   Python 7 6

4. video2dataset video2dataset Public

   Forked from iejMac/video2dataset

   Easily create large video dataset from video urls

   Python 1

5. Megatron-LLM Megatron-LLM Public

   Forked from epfLLM/Megatron-LLM

   distributed trainer for LLMs

   Python

6. data-PDF-pipeline data-PDF-pipeline Public

   PDF pipeline for creating training corpora (mainly for llm, multimodal and alignment horizontals)

   Python