FDEInterviews logo
AI Security, Privacy & Governance / 06
mediumMicrosoftScaleOpenAI

What is data/model poisoning, and how would you detect a backdoor introduced through fine-tuning or embedding data?

Poisoning questions sort candidates who've read about backdoors from those who can name where poison enters a real pipeline, and why the embedding store is the soft target nobody audits.

Updated Aug 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.

TL;DR: Poisoning corrupts what a model learns or retrieves; the dangerous variant is a backdoor that fires only on a trigger and survives normal evals. The entry points are pretraining (provider's problem), fine-tuning data, the embedding/RAG corpus (the soft target needing no training run), and model artifacts. You mostly cannot detect backdoors after the fact, so defend with provenance, least privilege on training and index stores, immutable versioning, and behavioral diff gates.

How to approach it

Structure beats depth here: define poisoning, enumerate the entry points in a modern LLM pipeline (there are more than the interviewer expects you to name), then get practical about detection, which is honestly hard, so the credibility move is emphasizing provenance and process controls over magical scanners.

A strong answer

Poisoning is corrupting what a model learns or retrieves so it misbehaves in attacker-chosen ways. The dangerous variant is the backdoor: the model behaves normally on everything except inputs containing a trigger, a token, phrase, or formatting quirk, where it flips to attacker-chosen behavior. Backdoors survive standard evaluation precisely because aggregate benchmarks never sample the trigger.

Entry points in a real enterprise pipeline, ordered by how often they're actually audited: (1) pretraining data, the provider's problem, your supply-chain question to them; (2) fine-tuning data, the live one for customers, especially when training on user-submitted content like support tickets, where an attacker can submit poisoned examples and wait; published work shows a few hundred crafted examples can implant behaviors, so the attack is cheap; (3) the embedding/RAG corpus, the softest target, because poisoning here needs no training run at all: plant a document in a wiki or shared drive the indexer crawls, optimize it to rank for target queries, and you've changed model behavior in production within one index refresh; (4) model artifacts themselves, a tampered checkpoint or a pickle-deserialization payload from a model hub, which is classic software supply chain (hash pinning, signed artifacts, safetensors over pickle).

Detection, honestly: for fine-tuning backdoors, you mostly can't grep for them after the fact, so defend the pipeline, provenance and access control on training data, anomaly screening before training (clustering to surface outlier examples, near-duplicate sprees from one source, label-flip patterns), canary evals after (behavioral diffs against the base model on a held-out safety and policy suite, plus fuzzing with candidate trigger patterns like rare tokens and weird unicode). For RAG poisoning, treat the corpus as untrusted by default: signed or allowlisted sources, anomaly detection on newly indexed documents that suddenly dominate retrieval for high-value queries, and retrieval logs you can audit when an answer looks steered.

The four entry points, ranked by how exposed they are versus how often anyone audits them:

rendering diagram…

The red node is the one an enterprise attacker uses this quarter: plant a document the indexer crawls and you change production behavior in one index refresh.

The red node deserves one concrete walk-through, because "plant a document" undersells how mundane the attack is. An attacker with nothing but wiki-edit or ticket-filing access creates a page titled "Updated wire transfer procedure (Q3)", written in exactly the phrasing employees use when they ask ("how do I send a wire," "vendor payment details"), with the attacker's account number embedded as the destination. The indexer crawls it on schedule; the page's tight topical match makes it rank for those queries; and the finance assistant now answers wire questions by faithfully summarizing the attacker's document, with a citation that makes it look more trustworthy. No model was trained, no system was breached in the classical sense, and every security control aimed at the model itself was irrelevant. The cleanup is the part that convinces people: you must find the document, purge its chunks, purge their embeddings, invalidate any caches holding the answer, and then audit the retrieval logs to learn who asked and what they were told, which is why the defenses are provenance and write-access control on the corpus, not anything clever at inference time.

The senior framing: poisoning is a supply-chain problem wearing an ML costume. The controls that work are the boring ones, provenance, least privilege on who can write to training and index stores, immutable versioning of datasets so you can diff and roll back, and behavioral regression gates between model versions.

What interviewers probe next

  • "Your customer fine-tunes on raw support tickets monthly. What's your one-paragraph risk brief?", Anyone who can file a ticket can write to your training set; mitigate with example-level filtering, source caps, behavioral diff gates before promotion, and a rollback path to the prior adapter.
  • "How is RAG poisoning different from prompt injection?", Injection manipulates one conversation; corpus poisoning persists and hits every user who triggers retrieval. Same payloads, very different blast radius and cleanup (you have to find and purge the document and its chunks and embeddings).
  • "What would you ask a model provider about pretraining poisoning?", Data governance and filtering practices, red-team results, and incident process; you're assessing their supply chain maturity, not auditing their corpus.

Common mistakes

  • Only discussing training-time poisoning and missing the embedding store, which is the vector an enterprise attacker would actually use this quarter.
  • Hand-waving "we'd scan the data for poison", against a competent backdoor, scanning has poor recall, and claiming otherwise signals you haven't tried.
  • Forgetting model-artifact supply chain (unsigned checkpoints, pickle payloads) when the question literally says "model poisoning."
  • No rollback story. The interviewer's hidden rubric usually includes "does the candidate version datasets and models so recovery is a revert, not an archaeology dig."
That one was free — and so are 10 answers per topic without an account. Signing in doubles that to 20, opens the Plus lessons in the courses, and remembers which topics you keep getting wrong.no card · Google sign-in · nothing to cancel
HOW DID IT GO?
0
READING SIGNED OUT

Signing in doubles your free answers, from 10 to 20 per topic, and the site starts remembering you: mastery per topic, bookmarks, and a next-focus recommendation. Free, no card.

Sign in free
UP NEXT ON YOUR JOURNEY
FEDITOR'S NOTE

The differentiator is naming the embedding/RAG corpus as an entry point, because poisoning there needs no training run at all: plant a document the indexer crawls and you've changed production behavior in one index refresh. Hand-waving 'we'd scan the data for poison' signals you haven't tried, since scanning has poor recall against a competent backdoor; the credible move is provenance, least privilege on who can write to training and index stores, and behavioral diff gates. The hidden rubric item is a rollback story: versioned datasets and models so recovery is a revert, not an archaeology dig.

DISCUSSION · 0

No comments yet — be the first to share your approach.