Design inference batching for a single GPU serving up to 100 inputs per batch while users wait synchronously. Maximize utilization.

The senior signal is naming the three knobs that define a batcher: max batch size, max wait time, and an early-flush on queue depth, then reasoning about the latency/throughput tradeoff between them with numbers. Strong candidates raise padding waste and length bucketing unprompted, and they isolate per-item failures so one bad input cannot fail the whole batch. The reserve follow-up is almost always 'now you have 8 GPUs', which tests whether they reach for continuous batching and a load balancer keyed on queue depth rather than round-robin. Watch for candidates who batch but never bound the queue, so a traffic spike turns into unbounded latency instead of backpressure.