FDEInterviews logo
System Design & Production Engineering / 05
easyPalantirRetoolMicrosoft

It's Friday evening at a customer site and they want a hotfix shipped now. How do you do it safely, and when do you refuse?

This isn't a deployment question, it's a judgment question wearing one. Interviewers are scoring whether you have a decision framework, a safety checklist, and the spine to say no to a customer politely.

Updated Aug 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.

TL;DR: Don't answer yes or no; triage by severity first. A P0 (data corrupting, money or safety at risk) justifies shipping tonight with the risk shrunk to a tiny diff, tested rollback, canary, and a second reviewer; anything less gets a safe mitigation now and the real fix Monday, in writing.

How to approach it

Resist answering "yes" or "no." The strong move is a triage framework: severity and blast radius first, then the conditions under which you'd ship, then the conditions under which you'd refuse and what you'd offer instead. Interviewers at Palantir and Microsoft ask this to test the consultant half of the FDE job: can you protect the customer from their own urgency without damaging the relationship?

A strong answer

First, clarify what "broken" means. Is this a P0 (production down, data corrupting, money or safety at risk) or an annoyance that feels urgent because an exec saw it? That single question decides everything. Data corruption that compounds over the weekend justifies real risk tonight; a cosmetic bug does not, because the expected cost of a botched Friday deploy (no one watching all weekend, skeleton staff, slower rollback) exceeds the cost of waiting until Monday.

rendering diagram…

If it warrants shipping, shrink the risk rather than the process:

  • Smallest possible diff: fix the bug, refuse the temptation to "also clean up" anything.
  • A tested rollback path before deploying: know the exact command to revert, and verify the previous artifact still deploys.
  • Feature flag or config kill-switch if the change can be gated, because turning it off beats redeploying at 2am.
  • Canary first: one instance or a small traffic slice, watch error rates and latency for 20 to 30 minutes, then roll forward.
  • A second pair of eyes, even a 15-minute review over a call. Solo Friday deploys are where careers go to die.
  • Stay on the hook: explicit monitoring plan for the next few hours, and tell the customer exactly who is reachable over the weekend.

If it doesn't warrant shipping, refuse with an alternative, not a flat no: "Deploying this tonight risks a weekend outage with nobody watching. Here's a mitigation that's safe now (we disable the broken path or apply a config workaround) and I'll ship the real fix Monday at 9am with full rollback coverage. Here's my number if anything escalates." You're declining the deploy, not the responsibility. Then put it in writing, a short email confirming severity assessment, the decision, and the plan, so Monday's narrative is calm and documented. The email is short enough to show whole, and having its shape memorized means you can send it from a hallway at 7pm:

Subject: Report-export bug: severity call and weekend plan

What's broken: CSV export fails for reports over 10k rows. Display and data
are unaffected; nothing is corrupted or lost.
Severity: P2. Annoying, not damaging; risk of a Friday deploy exceeds the
cost of the workaround.
Tonight: enabled the legacy export path as a workaround (verified working).
Monday: real fix ships at 9am with full rollback coverage; I'll confirm here.
Weekend contact: me, +xx-xxxx, if anything changes severity.

Five lines, no defensiveness, and every sentence is a commitment someone can check. If the situation later escalates, this email is what makes the escalation orderly instead of a blame hunt; if it stays quiet, it is what the customer's exec forwards upward as evidence their vendor has adults on site. The two minutes it takes are the highest-leverage two minutes of the evening.

What interviewers probe next

  • "The customer's VP insists, overriding your judgment." Restate the risk in business terms ("a failed deploy means your Monday-morning users are down"), offer the mitigation, and if they still insist, escalate to your own management rather than absorbing the risk silently. Executing under written protest is sometimes the job; doing it without flagging it never is.
  • "What if you can't reproduce the bug in staging?" That's close to disqualifying for a Friday ship; a fix you can't verify is a guess, and you say so.
  • "What changes if this is an air-gapped or regulated environment?" Change-control processes exist precisely for this; circumventing them at a government or healthcare customer is a fireable offense, not a hustle.

Common mistakes

Answering "never deploy on Friday" as dogma: it fails the P0 data-corruption case and reads as inflexible. Answering "the customer is always right": it fails the judgment test entirely. Listing deployment mechanics without ever asking how severe the bug is. And forgetting the relationship layer: no written summary, no Monday follow-through, no named on-call human, the things that make a customer trust you more after an incident than before it.

Key takeaways

  • Severity decides everything: P0 data or money risk earns a ship tonight; cosmetic urgency does not.
  • When you ship, shrink risk not process: smallest diff, tested rollback, canary, second reviewer.
  • Refuse with an alternative and a number, never a flat no; you decline the deploy, not the responsibility.
  • If a VP overrides you, escalate to your own management and put the protest in writing.
That one was free — and so are 10 answers per topic without an account. Signing in doubles that to 20, opens the Plus lessons in the courses, and remembers which topics you keep getting wrong.no card · Google sign-in · nothing to cancel
HOW DID IT GO?
0
READING SIGNED OUT

Signing in doubles your free answers, from 10 to 20 per topic, and the site starts remembering you: mastery per topic, bookmarks, and a next-focus recommendation. Free, no card.

Sign in free
UP NEXT ON YOUR JOURNEY
FEDITOR'S NOTE

Watch for the escalation perturbation: 'the customer's VP overrides you and insists.' The answer that scores is restating the risk in business terms, offering the safe mitigation, and then escalating to your own management rather than silently absorbing the risk; executing under written protest is sometimes the job, doing it without flagging it never is. Candidates who answer 'never deploy on Friday' as a rule fail the P0 data-corruption case just as hard as the ones who say 'the customer is always right.'

DISCUSSION · 0

No comments yet — be the first to share your approach.