The biggest trap in an ML system design interview is reaching for a model the moment you hear the question. The interviewer hands you a deliberately vague prompt ("design YouTube's recommender") not to test whether you know XGBoost, but to see whether you can turn an ambiguous problem into a system that actually ships.
This post uses the industry-standard 8-step framework (aligned with Aminian & Xu's ML System Design Interview), walked through the most common prompt — a recommender system — with the follow-up questions interviewers actually ask at each step. That's where the points are.
⏱️ Time budget: in a 45-minute interview, spend the first 5 minutes clarifying, ~5–7 minutes per block, and the last 5 minutes summarizing trade-offs.
Step 1 — Clarify Requirements
Don't start building. Pin down:
- Functional: Recommend to whom? What's the goal (CTR? watch time? retention?)? How many items per request?
- Non-functional: Scale (e.g., 1B users, 100M videos)? Latency (< 100ms)? How often does it refresh?
Recommender example: confirm the goal as "maximize long-term watch time," not just clicks (avoids clickbait). Drawing that distinction is itself a scoring point.
🎤 Interviewer follow-ups: "What's the actual business goal? What does success look like?" "What are the scale and latency constraints?"
Step 2 — Frame as an ML Task ⭐
The step most people skip, and the most important one. Translate the business goal into precise ML inputs/outputs.
Recommender example:
- Frame it as ranking: given (user, candidate items), predict an "engagement probability" per item, then sort by score.
- Usually two stages: Retrieval quickly narrows hundreds of millions down to a few hundred → Ranking finely scores those candidates.
- Nail down input (user + item + context features), output (P(engagement)), and label (watched / clicked).
🎤 Interviewer follow-ups: "Why ranking and not classification?" "How do you define the label — is a 3-second view a positive?"
Step 3 — Metrics
Split into offline and online — a guaranteed point of comparison.
- Offline: ranking metrics — NDCG@k, Recall@k, MAP, AUC/LogLoss. Don't use accuracy: recommendation is ranking, and the classes are heavily imbalanced.
- Online: CTR, average watch time, retention — validated via A/B test.
🎤 Interviewer follow-up (classic): "Offline metrics improved but online dropped — what now?" (Answer: check training-serving skew, an offline metric decoupled from the business goal, feedback loops, A/B setup issues.) "Why NDCG over accuracy?"
Step 4 — Data
- Source: mostly implicit feedback (clicks, watch time, skips).
- Positives/negatives: positive = watched / long view; negatives = impressions-not-clicked + random negative sampling.
- Imbalance: CTR may be a few percent — use negative sampling or class weights.
- Splitting: always split by time (predict the future from the past), never randomly — otherwise you leak.
🎤 Interviewer follow-ups: "How do you pick negatives? Is an unclicked impression really negative (maybe they didn't see it)?" "How do you handle imbalance?"
Step 5 — Feature Engineering
- User features: history, demographics, long- and short-term interests.
- Item features: category, content embeddings, stats (popularity, average completion rate).
- Context features: time, device, current session.
- High-cardinality IDs (user_id, video_id): use embeddings, not one-hot.
- Avoid leakage: features may only use information available before the prediction moment; watch the time window on aggregate features.
🎤 Interviewer follow-ups: "How do you feature-ize hundreds of millions of video IDs?" "How do you guarantee no future information leaks in?"
Step 6 — Model Architecture
Lead with a baseline, then go advanced — don't open with the most complex thing.
- Baseline: popularity / collaborative filtering (CF) to establish a comparison point fast.
- Retrieval layer: a two-tower model + ANN (approximate nearest neighbor) to pick candidates from hundreds of millions in milliseconds.
- Ranking layer: GBDT (e.g. LightGBM) or a deep model (Wide & Deep / DLRM-style) for fine-grained scoring.
🎤 Interviewer follow-ups: "Why not just run one big DNN over every video?" (Answer: scoring hundreds of millions of items blows the latency budget — hence two stages: cheap retrieval, then precise ranking.) "Why start with a baseline?"
Step 7 — Training
- Loss: pointwise (logloss), pairwise (BPR), or listwise — ranking often uses pairwise/listwise.
- Negative sampling: especially important for retrieval (in-batch negatives).
- Cold start: new users → popularity / demographics; new videos → content-feature embeddings.
- Retraining cadence: behavior shifts fast, so daily retraining or incremental updates are common.
🎤 Interviewer follow-up (classic): "How do you solve cold start — new users and new videos with no interaction data?" "How often do you retrain? Is online learning viable?"
Step 8 — Evaluation, Serving & Monitoring
- Evaluation: offline first (NDCG@k, etc.) → then A/B test for online metrics; ship to 100% only when both pass.
- Serving: retrieval candidates can be precomputed offline + ranked online in real time; respect the latency budget; use model compression (quantization, distillation) and caching.
- Monitoring: watch for data drift / model decay, prediction distributions, and online metrics; define retraining triggers; mind the recommender feedback loop (the model shapes impressions, which become training data).
🎤 Interviewer follow-ups: "With a 100ms latency cap, how does the ranking layer keep up?" "Performance slowly decays after launch — what do you do?" "How do you design the A/B test, and how long do you run it?"
One Table to Remember It All
| Step | Focus | Favorite follow-up |
|---|---|---|
| 1 Clarify | goal + scale + latency | What's the success metric? |
| 2 ⭐Frame as ML task | input/output/label, two stages | Why ranking? |
| 3 Metrics | offline NDCG@k vs online A/B | Offline up, online down — now what? |
| 4 Data | positives/negatives, split by time | Imbalance / leakage? |
| 5 Features | embeddings, no leakage | How to handle high-cardinality IDs? |
| 6 Model | baseline → retrieval + ranking | Why not one big DNN? |
| 7 Training | loss, negative sampling, cold start | How to solve cold start? |
| 8 Eval/Serve/Monitor | A/B, latency, drift | What about decay after launch? |
The One-Liner
"Frame the problem before you talk models." For any ML system design question, always run Steps 1–2 first (clarify + frame as an ML task) — turn the vague prompt into precise input/output/metric, then proceed. Do just this and you're already ahead of half the candidates.