THE RESEARCH LIBRARY · PAPERS FOR FDES
AI research, explained for the work.
50 papers on building, evaluating and serving AI. Understand the mechanism, examine the evidence, and see what changes when you put the idea into production.
The guided starting pathProduction foundations → retrieval → agents → serving
STAGE 01
Understand production work
Describe a customer workflow, its dependencies, what success means and who owns it after launch.
STAGE 02
Understand the model and grounding
Explain the architecture and build a basic question-answering app over documents.
STAGE 03
Measure retrieval and answers
Build a small eval set and test evidence placement, retrieval quality and faithfulness.
STAGE 04
Add controlled actions
Give an agent a few well-designed tools, clear stopping rules and useful feedback.
STAGE 05
Evaluate reliability
Write a rubric, check the judge against humans, and run repeated and perturbed tests.
STAGE 06
Understand customization and serving
Say when adaptation could help, and how context length and concurrency set the memory bill.
Find your next paper
Browse by topic, or search for an idea.
50 papers
Foundations
The ideas every system you deploy is built on.Hidden Technical Debt in ML Systems
Google's 2015 field guide to why ML systems are cheap to build and expensive to keep alive, and where the cost hides when nobody is looking.
Attention Is All You Need
The 2017 Google paper that introduced the Transformer: attention and position information in place of recurrent and convolutional layers.
GPT-3: Few-Shot Learners
The 2020 OpenAI paper that showed a big enough model can learn a task from a few examples in its prompt, no fine-tuning, and made prompting a discipline.
InstructGPT
The 2022 OpenAI paper that turned GPT-3 into an assistant with human feedback, and showed a 1.3B model people preferred to the 175B original.
Chain-of-Thought Prompting
The 2022 Google paper that showed examples with their working shown, not just their answers, can triple a large model's accuracy on multi-step problems.
RAG
The 2020 paper that named RAG: a generator paired with a searchable index, so a model's knowledge can be checked, cited and updated without retraining.
Sentence-BERT
The 2019 paper that made BERT usable for semantic search: one vector per sentence, compared by cosine, turning a 65-hour job into 5 seconds.
Lost in the Middle
The 2023 Stanford paper that showed models use evidence at the start and end of a long prompt far better than in the middle, sometimes worse than no evidence at all.
Building and evaluating LLM apps
Agents, pipelines, structured output and how to grade them.ReAct
The 2022 paper that defined the agent loop: think, call a tool, read the result, think again. Every agent framework you will deploy runs a version of it.
DSPy
The 2023 Stanford paper that treats an LLM pipeline as a program: declare what each call does, give a metric and a few examples, and let a compiler write the prompts.
Grammar-Constrained Decoding
The 2023 EPFL paper showing that masking a model's tokens with a grammar guarantees the output's shape, lifts accuracy without training, and has a trap: valid but empty.
LLM-as-a-Judge
The 2023 paper that made using a strong model to grade other models respectable, by measuring how often it agrees with people and exactly how it is biased.
Ragas
The 2023 paper that split RAG quality into three checks you can run without reference answers: is the context focused, is the answer grounded in it, does it address the question.
Retrieval, documents and memory
Getting the right context in front of the model, and keeping it.RAG or Long-Context LLMs?
The 2024 Google DeepMind study that found long context beats RAG on quality, RAG wins on cost, and a model that says when it cannot answer can route between them.
Late Chunking
The 2024 Jina AI paper that fixes chunks losing their context: run the whole document through the embedding model first, then cut the token vectors into chunks.
GraphRAG
Microsoft's 2024 paper on answering questions about a whole corpus, not one passage: build an entity graph, summarize its communities, then map-reduce over the summaries.
ColPali
The 2024 paper that skips PDF parsing: embed each page image with a vision-language model into many small vectors, and let tables, charts and layout count in search.
Qwen3-VL Embedding and Reranker
Alibaba's 2026 report on one embedding model and one reranker for text, images, document pages and video, and the trade-offs of dimensions, precision and reranking.
RAGPerf
The 2026 UIUC and IBM benchmark that profiles a RAG system stage by stage and finds where the time, memory and money really go, rarely where teams look.
Agentic Context Engineering (ACE)
The 2025 Stanford and SambaNova paper that improves agents by growing a detailed playbook of lessons in their context, with small edits instead of rewrites, and no weight updates.
Recursive Language Models
The 2025 MIT paper that stops stuffing long inputs into the context window: put the input in a variable, let the model write code to slice it and call itself on the pieces.
Mem0
The 2025 paper behind the Mem0 memory layer: extract facts from conversations, keep them current with add, update and delete, and trade a little accuracy for a lot of speed.
Agents, evaluation and security
What agents do in the real world, how they fail, and how they are attacked.SWE-agent
The 2024 Princeton paper that showed an agent's tools matter as much as its model: a purpose-built interface for searching, viewing and editing code beat a raw shell by seven points.
AI Agents That Matter
The 2024 Princeton critique of agent benchmarks: report cost next to accuracy, compare against simple baselines, keep a real held-out set, and stop mistaking leaderboards for procurement advice.
Why Multi-Agent Systems Fail
Berkeley read 1,642 traces from seven multi-agent frameworks and named 14 ways they fail. Most failures trace back to how the agents are organized and checked, not to the model inside them.
Measuring Agents in Production
Twenty deployment teams and 86 deployed systems describe how agents actually ship: frontier models prompted by hand, fixed workflows, short leashes and humans checking the output.
Reflexion
The 2023 paper that taught agents to learn from failure without training: after each failed attempt, the model writes itself a lesson and reads it on the next try.
SWE-bench
The benchmark that moved coding evaluation from toy functions to real repositories: 2,294 GitHub issues, each graded by the project's own tests. In 2023 the best model fixed under 2%.
τ-Knowledge
A support-agent benchmark built like a real deployment: 698 interlinked bank documents, tools you can only find by reading them, and a simulated customer. The best agent passed a quarter of tasks.
AI Agent Reliability
Princeton borrowed reliability engineering from aviation and nuclear power, turned it into twelve metrics for AI agents, and found that two years of accuracy gains bought only small reliability gains.
SkillsBench
Agent Skills, folders of instructions and scripts an agent loads on demand, lifted pass rates from 34% to 51% across 18 agent setups. Focused Skills helped most; self-written ones hurt.
ττ-bench (hyper-tau-bench)
A benchmark that turns an FDE engagement into the task: messy records, a client to interview, a flaky API and a serving budget. AI-built agents scored under a third of an expert's.
AgentDojo
The standard testbed for prompt injection against tool-using agents: 97 tasks, 629 attack cases, and a finding that the most capable models were often the easiest to hijack.
Indirect Prompt Injection Arena
464 red-teamers made 272,000 attempts to hijack 13 frontier agents and make them hide it. Every model fell at least sometimes, and attacks that beat the strongest model worked almost everywhere.
Customization and reasoning
Changing a model's behavior, and paying for it to think longer.LoRA
The 2021 Microsoft paper behind most fine-tuning today: freeze the model, learn each layer's change as two thin matrices, and match full fine-tuning of GPT-3 with 10,000× fewer trained parameters.
QLoRA
LoRA on top of a 4-bit base model: QLoRA fit a 65B fine-tune onto one 48 GB GPU with no measured loss, and showed that 9,000 good examples beat 450,000 mediocre ones.
DPO
The 2023 Stanford paper that replaced RLHF's reward model and PPO loop with one classification loss on preference pairs, and became the default way to tune models on human preferences.
Tülu 3
AI2's fully open post-training recipe: every dataset, script and evaluation used to turn Llama 3.1 base models into assistants that matched closed models, plus RL with verifiable rewards.
DeepSeek-R1
DeepSeek showed that reinforcement learning on checkable problems, with no human reasoning examples, teaches a model to think at length and reach o1-level math. Small distilled versions followed.
Scaling Test-Time Compute
When does thinking longer beat a bigger model? Spending inference compute by question difficulty used up to 4× less compute and beat a 14× larger model on questions within reach.
Serving, latency and cost
What makes inference fast, cheap and able to hold many users.FlashAttention
The 2022 Stanford kernel that made attention fast by counting memory trips, not math: exact attention computed in on-chip tiles, several times faster, with memory linear in sequence length.
PagedAttention and vLLM
The Berkeley paper behind vLLM: manage the KV cache like operating-system virtual memory, in small pages allocated on demand, and serve 2 to 4 times more requests on the same GPU.
AWQ
MIT's 4-bit weight quantization that asks the activations which weights matter, protects about 1% of channels by scaling instead of keeping them in 16-bit, and runs over 3× faster than FP16 on GPUs.
SGLang
A serving system for programs that call a model many times: it caches KV across requests in a radix tree, decodes fixed JSON text in one step, and ran agent workloads up to 6.4× faster.
DistServe
Prefill is compute-bound and decode memory-bound, so sharing one GPU makes each slow the other. DistServe splits them onto separate GPUs and served up to 7.4× more requests within latency targets.
LMCache
Treat the KV cache as data that outlives the request: store it beyond the GPU, move it in big chunks, and reuse it across queries and engines. With vLLM, up to 15× throughput.
RouteLLM
Train a small router on human preference data to send each query to a strong, costly model or a weak, cheap one. On MT Bench it reached 95% of GPT-4's score using GPT-4 for 13% of queries.
Beyond text
Vision, speech and the tabular baseline you should always try first.CLIP
Train an image encoder and a text encoder so each photo lands next to its own caption. The result classifies images from a list of class names, with no labeled training data.
Whisper
Train one transformer on 680,000 hours of web audio and its imperfect transcripts. With no benchmark fine-tuning, it made about half the errors of a LibriSpeech-trained model on other test sets.
XGBoost
Gradient boosted trees with a regularized objective, built-in handling of missing values and serious systems engineering. Still the first model to try on a customer's tables.
“Most cited” uses dated Semantic Scholar counts shown on each paper. Citation counts describe research influence, not a recommendation for your system.