README.md

Awesome AI Engineer Reads

🏗️ Building and Deploying LLMs

⚖️ Evaluating LLM + Retrieval

🔍 LLM + Retrieval

💻 LLM Applications

🧠 LLM Architectures and Models

💼 LLM Business

📊 LLM Data

🚀 LLM Deployment

🧩 LLM Embeddings

🔧 LLM Engineering

🏆 LLM Engineering Best Practices

⚖️ LLM Ethics and Governance

🔬 LLM Evaluation

🎫 LLM Events

🤯 LLM Hype

🤔 LLM Inference

🏗️ LLM Infrastructure

📢 LLM Marketing

📰 LLM Newsletters

🔢 LLM Numbers

👀 LLM Observability

💬 LLM Opinions and Critiques

📝 LLM Prompting

📔 LLM Research and Publications

🎣 LLM Retriever Models

💰 LLM Startups and Funding

📗 LLM Tutorials and Courses

🔓 Open LLM Models

🔧 Open LLM Tools

🖥️ Self-Hosted LLMs

🛠️ Tools and Frameworks

🏋️ Training and Fine-tuning LLMs

❔ Uncategorized

Building and Deploying LLMs

• Scaling Kubernetes to 7,500 nodes
• All the Hard Stuff Nobody Talks About when Building Products with LLMs
• All the Hard Stuff Nobody Talks About when Building Products with LLMs
• Building LLM applications for production
• Efficiently Scaling and Deploying LLMs
• LLMs in Production - Part III

Training and Fine-tuning LLMs

• How to train your own Large Language Models
• Training Compute-Optimal Large Language Models
• Opt-175B Logbook
• Fine-Tuning LLMs: Best Practices and When to Go Small
• Finetuning Large Language Models
• GPT-3.5 Turbo fine-tuning and API updates
• Fine-tuning in Your Voice Webinar

LLM Architectures and Models

• A Survey of Large Language Models
• LLaMA: Open and Efficient Foundation Language Models
• Introducing Code Llama, a state-of-the-art large language model for coding
• Spread Your Wings: Falcon 180B is here
• BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

Tools and Frameworks

• LangChain: Enabling LLMs to Use Tools
• Langchain Tutorials
• LangChain cookbook
• fairseq2
• seamless_communication

LLM Applications

• Automatic Generation of Visualizations and Infographics with LLMs
• Introducing AudioCraft: A Generative AI Tool For Audio and Music
• An example of LLM prompting for programming
• Human-centric & Coherent Whole Program Synthesis aka your own personal junior developer
• GPT Engineer

LLM Evaluation

• Holistic Evaluation of Language Models
• chatgpt-evaluation-01-2023
• Evaluating chatGPT
• PromptBench: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts

LLM Prompting

• Prompt Engineering
• "Prompt injection explained, with video, slides, and a transcript"
• Delimiters won’t save you from prompt injection
• You probably don't know how to do Prompt Engineering
• ChatGPT Prompt Engineering for Developers
• Learn Prompting

LLM Research and Publications

• A Survey of Large Language Models
• Challenges and Applications of Large Language Models
• Large Language Models as Optimizers
• Multimodal Foundation Models: From Specialists to General-Purpose Assistants
• Scaling Laws for Neural Language Models
• Language Models are Unsupervised Multitask Learners
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• Training language models to follow instructions with human feedback

LLM Embeddings

• FlagEmbedding
• One Embedder, Any Task: Instruction-Finetuned Text Embeddings
• OpenAI GPT-3 Text Embeddings - Really a new state-of-the-art in dense text embeddings?
• Getting creative with embeddings
• What are embeddings?

LLM Retriever Models

• Vector Databases and Large Language Models
• Large Language Models with Semantic Search
• MultiVector Retriever
• Fine-Tuning LLaMA for Multi-Stage Text Retrieval

LLM plus Retrieval

• Making Retrieval Augmented Generation Better with @jamesbriggs
• Harnessing Retrieval Augmented Generation With Langchain
• Knowledge Retrieval Architecture for LLM’s (2023)
• How do domain-specific chatbots work? An Overview of Retrieval Augmented Generation (RAG)
• LangChain "Advanced Retrieval" Webinar
• TWIML-RAG - a TWIML generative_ai community project.

Evaluating LLM plus Retrieval

• Evaluation & Hallucination Detection for Abstractive Summaries
• Building And Troubleshooting An Advanced LLM Query Engine
• Advanced RAG 02 - Parent Document Retriever

LLM Inference

• Easy-to-use headless React Hooks to run LLMs in the browser with WebGPU. As simple as useLLM().
• Inference Experiments with LLaMA v2 7b
• OpenAI's Code Interpreter in your terminal, running locally
• Discover, download, and run local LLMs

LLM Data

• Want High Performing LLMs? Hint: It is All About Your Data
• How LlamaIndex Can Bring the Power of LLM's to Your Data
• How to Create Custom Datasets To Train Llama-2
• Data Copilot

LLM Observability

• AI Observability & Evaluation - Evaluate, troubleshoot, and fine tune your LLM, CV, and NLP models in a notebook.

LLM Opinions and Critiques

• Why Chatbots Are Not the Future

LLM Tutorials and Courses

• Finetuning Large Language Models
• Large Language Models with Semantic Search
• How Business Thinkers Can Start Building AI Plugins With Semantic Kernel
• LLM Bootcamp - Spring 2023
• LLM Bootcamp - Spring 2023
• Generative AI for Beginners - A Course
• Artificial Intelligence for Beginners - A Curriculum
• Introduction to Deep Learning

LLM Engineering

• Reflections on AI Engineer Summit 2023
• AI Engineer Summit - Building Blocks for LLM Systems & Products

LLM Engineering Best Practices

• All the Hard Stuff Nobody Talks About when Building Products with LLMs
• Building Defensible Products with LLMs
• What Is ChatGPT Doing ... and Why Does It Work?
• Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges

Open LLM Models

• LLaMA: Open and Efficient Foundation Language Models
• BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

Open LLM Tools

• LangChain: Enabling LLMs to Use Tools
• Langchain Tutorials
• LangChain cookbook
• fairseq2
• seamless_communication

Self-Hosted LLMs

• Why host your own LLM?
• How is LLaMa.cpp possible?
• Discover, download, and run local LLMs

LLM Deployment

• Deploying AI-driven apps on Vercel
• Solving the Last Mile Problem of Foundation Models with Data-Centric AI
• Taking LangChain Apps to Production with LangChain-serve

LLM Infrastructure

• Age of Industrialized AI
• DevTools for Large Language Models: Unlocking the Future of AI-Driven Applications
• Large Language Model at Scale

LLM Events

• "AI Agents Hackathon Develop your own AI agent within 48 hours!"

LLM Numbers

• Numbers every LLM Developer should know

LLM Hype

• Anti-hype LLM reading list
• ChatGPT Resources

LLM Newsletters

• ✨ Flashier Attention, 🤐 Gzip Classifiers

LLM Ethics and Governance

• Constitutional AI: Harmlessness from AI Feedback
• A guidance language for controlling large language models.

LLM Startups and Funding

• The AI Startup Litmus Test
• Generative AI Strategy
• Lessons from 139 YC AI startups (S23)

LLM Business

• 2023: The State of Generative AI in the Enterprise
• Takeaways & lessons from 250k+ LLM calls on 100k corporate docs

LLM Marketing

• Your Success with Generative AI May Come Down to These UX Decisions

Uncategorized

https://github.com/yeagerai/yeagerai-agent
https://github.com/homanp/superagent
https://github.com/homanp/langchain-ui
https://github.com/0xpayne/gpt-migrate
https://github.com/shinework/photoshot
https://arxiv.org/abs/2201.11903
https://arxiv.org/abs/2210.03629
https://arxiv.org/abs/2303.11366
https://arxiv.org/abs/2305.10601
https://news.ycombinator.com/item?id=36645575
https://github.com/prefecthq/marvin
https://github.com/eth-sri/lmql
https://arxiv.org/pdf/2212.06094.pdf
https://github.com/mlc-ai/web-llm
https://github.com/Atome-FE/llama-node
https://github.com/go-skynet/LocalAI
https://localai.io
https://www.cursor.so/blog/llama-inference
https://www.geoffreylitt.com/2023/03/25/llm-end-user-programming.html
https://every.to/chain-of-thought/what-comes-after-saas
https://github.com/axilla-io/axgen
https://www.axilla.io/
https://github.com/f/awesome-chatgpt-prompts/blob/main/README.md
https://magrawala.substack.com/p/unpredictable-black-boxes-are-terrible
https://dl.acm.org/doi/10.1145/267505.267514
https://www.youtube.com/@gklitt/videos
https://www.inkandswitch.com
https://idl.cs.washington.edu/files/2019-AgencyPlusAutomation-PNAS.pdf
https://simonwillison.net/2023/Mar/27/ai-enhanced-development/
https://www.robinsloan.com/notes/home-cooked-app/
https://dl.acm.org/doi/10.1145/2593882.2593896
https://wattenberger.com/thoughts/boo-chatbots
https://www.geoffreylitt.com/2023/07/25/building-personal-tools-on-the-fly-with-llms.html
https://web.mit.edu/6.031/www/sp22/
https://www.youtube.com/watch?v=bJ3i4K3hefI
https://github.com/tianlinxu312/Everything-about-LLMs
https://arxiv.org/abs/2311.04205
https://arxiv.org/abs/2309.04269
https://arxiv.org/abs/2310.11511
https://arxiv.org/pdf/2310.07064
https://arxiv.org/abs/2309.15217
	https://arxiv.org/abs/2304.08354
	https://arxiv.org/abs/2203.11171
 	https://arxiv.org/abs/2310.06692
 	https://arxiv.org/abs/2310.05029
 	https://arxiv.org/abs/2005.11401
  	https://arxiv.org/abs/2212.10071
   https://arxiv.org/abs/2301.12726
   https://arxiv.org/abs/2305.01879
   https://arxiv.org/abs/2305.02301
   https://arxiv.org/abs/2212.00193
   https://arxiv.org/abs/2305.13888
   https://arxiv.org/abs/2306.09299
   https://arxiv.org/abs/2207.00112
   https://arxiv.org/abs/2307.00526
   https://arxiv.org/abs/2106.09685
   https://arxiv.org/abs/2210.07558
   https://arxiv.org/abs/2311.12023
   https://arxiv.org/abs/2311.11696
   https://arxiv.org/abs/2311.09179
   https://arxiv.org/abs/2311.08598
   https://arxiv.org/abs/2311.05556
   https://cset.georgetown.edu/publication/techniques-to-make-large-language-models-smaller-an-explainer
   https://arxiv.org/abs/2304.01089
   https://arxiv.org/abs/2304.07493
   https://arxiv.org/abs/2304.09145
   https://arxiv.org/pdf/2306.02272
   https://arxiv.org/abs/2307.09782
   https://arxiv.org/abs/2303.08302
   https://arxiv.org/abs/2306.07629
   https://arxiv.org/abs/2307.13304
   https://arxiv.org/abs/2308.15987v1
   https://arxiv.org/abs/2309.01885
   https://arxiv.org/abs/2309.02784
   https://arxiv.org/abs/2309.05516
   https://arxiv.org/abs/2308.13137
   https://arxiv.org/abs/2306.08543
   https://arxiv.org/abs/2306.13649
   https://arxiv.org/abs/2305.14864
   https://arxiv.org/abs/2301.00234
   https://arxiv.org/abs/2301.11916
   https://arxiv.org/abs/2212.10670
   https://arxiv.org/abs/2210.06726
   https://arxiv.org/abs/2212.08410
   https://arxiv.org/abs/2305.14152
   https://arxiv.org/abs/2305.14314
   https://arxiv.org/abs/2210.17323
   https://arxiv.org/abs/2306.00978
   https://www.ben-evans.com/benedictevans/2023/10/5/unbundling-ai
   https://www.coatue.com/blog/perspective/ai-the-coming-revolution-2023
   https://arxiv.org/pdf/2308.07633
   https://arxiv.org/abs/2301.00774
   https://arxiv.org/abs/2305.18403
   https://arxiv.org/abs/2306.11695
   https://arxiv.org/abs/2305.11627
   https://arxiv.org/abs/2305.17888
   https://arxiv.org/abs/2305.14152
   https://arxiv.org/abs/2206.09557
   https://arxiv.org/abs/2208.07339
   https://arxiv.org/abs/2206.09557
   https://arxiv.org/abs/2208.07339
   https://arxiv.org/abs/2206.01861
   https://arxiv.org/abs/2211.10438
   https://arxiv.org/abs/2305.14152
   https://arxiv.org/abs/2305.17888
   https://arxiv.org/abs/2306.03078
   https://arxiv.org/pdf/2311.10122
   https://arxiv.org/pdf/2311.10093
   https://arxiv.org/pdf/2311.08263
   https://arxiv.org/pdf/2311.07575
   https://arxiv.org/pdf/2311.06783
   https://arxiv.org/pdf/2311.09210v1
   https://arxiv.org/pdf/2311.10709
   https://arxiv.org/pdf/2311.07361
   https://arxiv.org/pdf/2311.07989
   https://arxiv.org/pdf/2311.02462
   https://arxiv.org/abs/2311.05232
   https://arxiv.org/pdf/2311.03285v1
   https://arxiv.org/pdf/2311.05556
   https://arxiv.org/pdf/2311.05437
   https://arxiv.org/pdf/2311.05348
   https://arxiv.org/pdf/2311.04257
   https://arxiv.org/pdf/2311.04219
   https://arxiv.org/pdf/2311.03356
   https://arxiv.org/pdf/2311.05657
   https://arxiv.org/pdf/2311.05997
   https://arxiv.org/pdf/2311.04254
   https://arxiv.org/pdf/2311.03301
   https://arxiv.org/pdf/2311.04400
   https://arxiv.org/pdf/2311.04145
   [Universal Language Model Fine-tuning for Text Classification](https://arxiv.org/pdf/1801.06146.pdf)
   https://idratherbewriting.com/blog/writing-full-length-articles-with-claude-ai
   https://www.oliverwyman.com/our-expertise/insights/2023/nov/impact-of-artificial-intelligence-in-financial-services.html
   https://www.youtube.com/watch?v=zjkBMFhNj_g
   https://towardsdatascience.com/recreating-andrej-karpathys-weekend-project-a-movie-search-engine-9b270d7a92e4
   https://streamlit.io/generative-ai
   https://www.youtube.com/watch?app=desktop&v=1RxOYLa69Vw
   https://www.bentoml.com/blog/announcing-open-llm-an-open-source-platform-for-running-large-language-models-in-production
   https://www.pinecone.io/learn/chunking-strategies/
   https://blog.llamaindex.ai/evaluating-the-ideal-chunk-size-for-a-rag-system-using-llamaindex-6207e5d3fec5
   https://amatriain.net/blog/hallucinations
   https://amatriain.net/blog/PromptEngineering
   https://realpython.com/chromadb-vector-database/
   https://github.com/zilliztech/VectorDBBench
   https://qdrant.tech/benchmarks/
   https://docs.ragas.io/en/latest/index.html
   https://towardsdatascience.com/10-ways-to-improve-the-performance-of-retrieval-augmented-generation-systems-5fa2cee7cd5c
   https://medium.com/@kelvin.lu.au/disadvantages-of-rag-5024692f2c53
   https://towardsdatascience.com/the-untold-side-of-rag-addressing-its-challenges-in-domain-specific-searches-808956e3ecc8
   https://www.anyscale.com/blog/a-comprehensive-guide-for-building-rag-based-llm-applications-part-1
   https://github.com/BuilderIO/gpt-crawler
   https://www.secondstate.io/articles/mistral-7b-instruct-v0.1/
   https://github.com/stas00/ml-engineering/tree/master
   https://arxiv.org/abs/2303.12712
   https://arxiv.org/abs/2203.15556
   https://arxiv.org/abs/2309.00267
   https://github.com/karpathy/llama2.c/blob/master/run.c
   https://github.com/ggerganov/llama.cpp
   https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models/
   https://arxiv.org/abs/2203.02155
   https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf
   https://arxiv.org/abs/1706.03762
   https://old.reddit.com/r/LocalLLaMA/comments/1atquor/im_open_sourcing_our_rag_backend_our_cqh_gql_chs/
   https://ravinkumar.com/GenAiGuidebook/
   https://towardsdatascience.com/advanced-retrieval-augmented-generation-from-theory-to-llamaindex-implementation-4de1464a9930
   https://srush.github.io/annotated-mamba/hard.html