How to Design a Recommendation Engine
Learn how to design a recommendation engine: offline training, candidate generation, fast ranking, cold-start, and feedback loops.
Expected Interview Answer
A recommendation engine splits work into an offline pipeline that trains models and precomputes candidate item lists from historical interaction data, and an online serving path that retrieves a small candidate set fast, ranks it with a lightweight model under a strict latency budget, and logs results to feed the next training cycle.
Offline, batch jobs process historical interactions (clicks, purchases, watch time) to train collaborative-filtering or embedding-based models, and periodically materialize candidate lists (e.g., "users who liked X also liked Y") into a fast-lookup store, since training a deep model per request is far too slow for real-time serving. Online, a request first goes through candidate generation — pulling a few hundred plausible items from precomputed similarity indexes, trending lists, or an approximate-nearest-neighbor lookup over embeddings — then a ranking model scores and orders that much smaller candidate set using richer features (recency, context, user session signals), all within a tight latency budget (typically under 100-200ms). Cold-start users or items with little history fall back to popularity-based or content-based recommendations rather than an empty result. Every impression and click is logged for both offline model retraining and online A/B testing, closing the loop between what was recommended and what users actually engaged with, and the whole system needs to be resilient to always return something reasonable (e.g., popular items) if the personalized path fails or times out.
- Separating offline training from online serving keeps recommendation latency low
- Candidate generation followed by ranking avoids scoring the entire catalog per request
- Fallback strategies handle cold-start users/items gracefully
- Logging impressions and clicks closes the feedback loop for continuous model improvement
AI Mentor Explanation
A recommendation engine is like a franchise’s scouting department preparing a shortlist of promising players overnight from historical performance data, rather than analyzing every player in the world during a live auction. On auction day, the team only reviews that pre-narrowed shortlist and quickly ranks a handful of top picks using fresh context like current squad needs, mirroring candidate generation followed by fast ranking. A completely unknown player with no track record still gets considered via general scouting heuristics rather than being ignored entirely, mirroring a cold-start fallback. Every bid and pick outcome is recorded to refine next season’s scouting models, closing the feedback loop.
Step-by-Step Explanation
Step 1
Train and precompute offline
Batch jobs process historical interactions to train models and materialize candidate lists (similar items, popular items) into a fast-lookup store.
Step 2
Generate candidates online
On request, pull a few hundred plausible items from precomputed indexes or approximate-nearest-neighbor lookups over embeddings — never score the whole catalog live.
Step 3
Rank the small candidate set
A lightweight ranking model scores and orders the candidates using richer, fresher signals (session context, recency) within a tight latency budget.
Step 4
Handle cold-start and log feedback
Fall back to popularity/content-based recommendations for new users/items, and log every impression and click to retrain the model.
What Interviewer Expects
- Splits the system into offline training/precomputation and online low-latency serving
- Describes candidate generation followed by ranking, not scoring the full catalog live
- Addresses cold-start users and items with a fallback strategy
- Mentions closing the feedback loop by logging impressions/clicks for retraining and A/B testing
Common Mistakes
- Proposing to run a full model inference over the entire catalog on every request
- Ignoring cold-start users/items entirely
- Not separating offline batch training from the online serving path
- Forgetting to log outcomes, so the model never improves from real user feedback
Best Answer (HR Friendly)
“A recommendation engine does the heavy lifting of analyzing everyone's behavior ahead of time, offline, so that when someone actually opens the app, it only has to quickly narrow down and rank a small set of already-prepared suggestions instead of thinking from scratch. New users or new items without much history still get sensible default suggestions, and everything people click or ignore gets fed back in to make future recommendations better.”
Code Example
def get_recommendations(user_id, limit=20):
user_profile = user_embedding_store.get(user_id)
if user_profile is None or user_profile.interaction_count < MIN_INTERACTIONS:
# Cold-start fallback: no personalized embedding yet
return popularity_index.top_items(limit=limit)
# Candidate generation: cheap approximate nearest-neighbor lookup,
# never a full scan or live model inference over the whole catalog.
candidates = ann_index.query(user_profile.embedding, top_k=300)
# Ranking: lightweight model scores the small candidate set with
# fresh contextual features, within the request latency budget.
scored = ranking_model.score(
user_id=user_id,
candidates=candidates,
context={"time_of_day": current_hour(), "device": request.device},
)
top_items = sorted(scored, key=lambda c: c.score, reverse=True)[:limit]
log_impressions(user_id, top_items) # feeds offline retraining + A/B analysis
return top_itemsFollow-up Questions
- How would you evaluate a new ranking model before rolling it out to all users?
- How would you keep candidate generation indexes fresh as new items are added continuously?
- What latency budget would you set for candidate generation versus ranking, and why?
- How would you detect and mitigate feedback loops that over-recommend already-popular items?
MCQ Practice
1. Why does a recommendation engine split candidate generation from ranking instead of scoring the whole catalog directly?
Candidate generation cheaply narrows the space to a few hundred plausible items so the more expensive ranking model only scores a manageable set within the latency budget.
2. What is the cold-start problem in a recommendation engine?
Without enough historical interactions, personalization signals are weak, so systems fall back to popularity- or content-based recommendations.
3. Why log every impression and click from the recommendation engine?
Logged interactions become the training data and evaluation signal for improving future versions of the recommendation model.
Flash Cards
Why split offline and online in a recommendation engine? — Training/precomputation is slow and runs in batch offline; serving must be fast, so it only retrieves and ranks precomputed candidates.
What is candidate generation? — Cheaply narrowing the full catalog down to a few hundred plausible items before detailed ranking.
How is cold-start handled? — Fall back to popularity- or content-based recommendations for users/items with little history.
Why log impressions and clicks? — To close the feedback loop for retraining models and running A/B tests.