Prompt Engineering
Prompt engineering is the practice of shaping a model's behavior through the instructions, examples, and format constraints you give it, before reaching for retrieval or fine-tuning. FDE loops test it because it is the cheapest, fastest lever you have, and a candidate who can make a model reliable with a well-structured prompt has saved a project weeks of unnecessary infrastructure.
Transcript
Your prompt works in testing, and breaks on 5% of real inputs. What do you do? The weak answer proposes fine-tuning for a problem that a three-line system prompt would have solved, and skips past prompting, which is the cheapest and fastest lever you have.
A production prompt has layers. Confusing them causes flaky behavior. The system prompt sets the role, the tone and the hard rules that should hold across every turn, while the user prompt carries this turn's request. Instructions say what to do. Few-shot examples show it, for when a task is easier to show than to describe, and the output format spells out the exact shape you expect back.
State instructions as commands. "It would be nice to get a summary" is a wish. "Return a JSON object with keys summary and risk level" is a command, and it wins. For the format, show a literal template. It's far more reliable than describing the shape in prose.
Here's one that classifies support tickets by urgency. It opens with a system line that pins the role and the output discipline. It gives an explicit label set, to tell the model which categories to use; validate the returned label. Then a literal format line.
Last, one example that demonstrates the reasoning you want, and then the ticket to classify. Every one of those moves reduces the risk of a bad output.
Why only one example? Because the model imitates the pattern. Show it three tickets that all mention payments, and it leans toward payment-shaped answers on a ticket about shipping. Two or three good ones usually beat ten mediocre ones.
Now the 5% failure. Collect the failures, the actual inputs, and look for the pattern across all of them rather than fixing them one at a time. Make the instruction a command. Add one example covering that case, pin the output format, and re-run the whole set, old cases too, until it holds.
Here's a hypothetical run. The prompt handles 95 of 100 held-out tickets, and you read the five it got wrong. Three describe two problems, one urgent and one trivial, and the model rated them "medium" as a compromise, while two are angry messages about cosmetic issues that it rated "high" because of the tone. That's two patterns, not five bugs. Add the missing rule: rate the most severe issue. Sharpen the existing one, ignore tone, plus one angry low-urgency example. Then run all hundred again, because a fix for one case can break another.
Prompting stops being enough on two fronts. When the model needs facts it doesn't have, like this week's inventory, that's the job of retrieval, and when its ingrained behavior or style has to change, that points to fine-tuning. Format, clarity and tone stay with the prompt. So the order is prompting, then RAG, then fine-tuning, which is also cheapest to most expensive.
So when the prompt breaks on real traffic, don't blame the model and don't jump to training. Collect the failures, find the pattern, and fix it with a clearer instruction, one example or a format spec. The triage prompt and the repair loop are on fdeinterviews.com.
TL;DR: Prompt engineering is making a model reliable by being explicit: a system prompt that sets role and rules, clear instructions, a few examples when the task is fuzzy, and a stated output format. Reach for it first because it costs nothing to iterate on. Move to RAG when the model needs facts it does not have, and to fine-tuning when you need to change its default behavior, not its knowledge.
The pieces of a prompt
Most production prompts have a few distinct layers, and confusing them is a common source of flaky behavior.
- System prompt. Sets the model's role, tone, and hard rules that should hold across every turn. "You are a support agent for Acme. Answer only from the provided context. If the answer is not there, say you do not know." This is the right place for constraints, not the user turn.
- User prompt. The actual request or question for this turn.
- Instructions. What to do, stated as commands rather than wishes. "Return a JSON object with keys
summaryandrisk_level" beats "it would be nice to get a summary." - Few-shot examples. When the task is hard to describe but easy to demonstrate (a specific extraction format, a particular tone), show two or three input-output pairs. The model imitates the pattern. Zero-shot (instructions only) is fine for clear tasks; reach for examples when zero-shot output drifts from what you want. The imitation is the reason to keep examples few: a model shown three tickets that all mention payments will lean toward payment-shaped answers on a ticket about shipping, because the examples are part of the pattern it is completing, not a separate instruction it reads and sets aside.
- Output format constraints. Specify the exact shape you expect. Showing a literal template of the JSON or the structure you want is far more reliable than describing it in prose.
A worked example
Suppose you want to classify support tickets by urgency. A vague prompt ("how urgent is this ticket?") gives you a paragraph of prose you then have to parse. A structured one removes the ambiguity:
System: You triage support tickets. Output ONLY valid JSON.
Instructions: Classify the ticket's urgency as "low", "medium", or "high".
Base it on customer impact and time sensitivity only.
Format:
{"urgency": "low|medium|high", "reason": "<one sentence>"}
Example:
Ticket: "Login page returns 500 for all users."
{"urgency": "high", "reason": "Total outage affecting every user."}
Ticket: "Can you change the font on my invoice?"
Notice the moves: a system prompt that pins the role and the output discipline, an explicit label set that tells the model which categories to use, a literal format line, and one example that demonstrates the reasoning you want. Each one reduces the risk of a bad output. Instructions do not enforce a schema: validate the returned fields and labels before downstream use.
Now run the repair loop on it, with a hypothetical failure set of the kind the diagram describes. Suppose the prompt handles 95 of 100 held-out tickets and you read the five it got wrong. Three are tickets that describe two problems, one urgent and one trivial, and the model rated them "medium" as a compromise. Two are angry messages about cosmetic issues that the model rated "high" because of the tone. That is two patterns, not five bugs, and each maps to one move. For the first, an instruction the prompt did not have: "If the ticket contains more than one issue, rate the most severe one." For the second, the existing instruction made sharper, "Base it on customer impact and time sensitivity only; ignore tone," plus one example of an angry low-urgency ticket rated "low". Then the whole hundred run again, not the five, because a fix that repairs the angry-ticket case can break the outage case, and you only find out by rerunning what already worked.
| What the failures had in common | The move | Why it works |
|---|---|---|
| Two issues in one ticket, rated as a compromise | Add the rule the prompt was missing | The model had no instruction for the case, so it improvised |
| Tone read as urgency | Sharpen the existing rule and add one counterexample | The rule was present but underspecified; the example shows the boundary |
| Prose wrapped around the JSON | Put the literal template last and say "output only the object" | Format instructions closest to the output are followed most reliably |
When prompting is enough, and when it is not
The spine above is what to do when it is nearly enough, which is the follow-up an interviewer holds back: the prompt works in testing and breaks on real traffic.
Good prompting solves more than people expect. If the task is "rewrite this in a friendlier tone" or "extract these five fields," a clear prompt with an example is usually the whole answer, and adding RAG or fine-tuning would be wasted effort.
Matching the problem to the lever keeps you on the cheap end:
| Problem | Best tool |
|---|---|
| Missing or changing facts | RAG |
| Ingrained behavior or style must change | Fine-tuning |
| Output format, clarity, or tone | Prompting |
It stops being enough on two fronts. When the model needs information it was never trained on or that changes over time, your company's pricing, this week's inventory, no prompt can supply facts the model does not have; that is RAG's job. When you need to change the model's ingrained behavior or output style at a level that examples cannot reach consistently, or you have many examples and need lower latency than stuffing them in the prompt allows, that points to fine-tuning. The order to try them is prompting, then RAG, then fine-tuning, because that is also the order from cheapest to most expensive to build and maintain.
Why interviewers probe this
It reveals whether you reach for the simplest tool first. The weak signal is a candidate who proposes fine-tuning for a problem a three-line system prompt would solve. The follow-up they hold back is "your prompt works in testing but breaks on 5% of real inputs, what do you do?" The strong answer is concrete: collect the failures, look for the pattern, make the instruction more explicit, add an example covering that case, and constrain the output format, rather than immediately blaming the model or jumping to training.
Common misconceptions
- "Prompting is just typing a question." Production prompting is interface design: role, instructions, examples, and format, each chosen to remove a failure mode.
- "More examples always help." Examples cost tokens and can bias the model toward their specifics. Two or three good ones usually beat ten mediocre ones.
- "Politeness or magic phrases improve accuracy." Explicitness and structure improve accuracy. Studies that measure phrasing effects find them small and inconsistent across models and tasks, so begging or threatening the model is not a lever you can rely on; a stated rule and a stated format are.
- "If the prompt fails, fine-tune." Most prompt failures are fixed by clearer instructions, a format spec, or one targeted example, all far cheaper than training.
Key takeaways
- Separate the layers: system prompt for role and rules, user prompt for the request, plus explicit instructions and a stated output format.
- Use few-shot examples when a task is easier to show than to describe; keep them few and representative.
- Constrain the output format with a literal template, especially when something downstream must parse the result.
- Try prompting first, RAG when facts are missing or change, fine-tuning when behavior or format must change; that order is also cheapest to most expensive.