ML & MLOps

Inside a Recommendation Engine

See why candidate retrieval, ranking, eligibility and diversity rules are separate stages, and how exposure-aware feedback supports later learning.

Reference architecture - illustrative, not a client deployment

Show retrieval, ranking and business filtering as separate stages.

Complexity
Intermediate
Drawing
Supplied
  • Recommendations
  • Ranking
  • Retrieval
  • Personalization
  • Machine Learning

Overview

This reference architecture is illustrative. It is not a record of a client deployment or a promise of business uplift.

A recommendation request should not score an entire catalog with one giant model. Offline preparation learns representations or other retrieval assets from authorized behavior events and catalog data, then refreshes the candidate index and approved ranking model. Online candidate generation narrows a large catalog to a manageable set. Ranking uses richer request, user and item context to order that smaller pool. Deterministic eligibility and diversity rules then remove unavailable, restricted or repetitive items before anything is shown.

Google's recommendation guidance describes the same broad separation between candidate generation, scoring and reranking. It also warns that retrieval choices can favor popular items. The architecture therefore records which items were actually exposed and distinguishes that exposure from a click, purchase or other outcome. Future learning needs both; treating every unclicked catalog item as a negative would misrepresent what the user could see.

A concrete example

A signed-in reader opens an article. Candidate generation retrieves related topics and items based on authorized context. The ranking model orders them using the current page and recent interactions. Rules remove unavailable material, enforce age or regional eligibility, and limit near-duplicates. The service logs the displayed positions. A later click is joined to that exposure as an observed outcome. For a new anonymous visitor with no history, the system uses an explicit popular or content-based baseline and still applies the same eligibility rules.

Common misconception

"The candidate generator already has similarity scores, so it can provide the final ranking." Candidate scores from different sources may not be comparable, and final ranking can use context that broad retrieval cannot afford. Business constraints remain a separate deterministic step.

Diagram & Walkthrough

Choose a declared scenario and step, or select a component by pointer, touch or keyboard. Playback starts only when you press Play and pauses when this panel leaves the screen.

100%
Inside a Recommendation Engine A recommendation architecture separating offline preparation, online candidate generation, ranking, eligibility and diversity filtering, cold-start fallback, and outcome feedback. Inside a Recommendation Engine Retrieval, ranking, and business filtering are separate stages. OFFLINE PREPARATION ONLINE SERVING authorized request → eligible results FEEDBACK Historical interactions Embeddings + index Approved ranking model Authorized context Candidate items Ranked candidates Eligible diverse results Exposure context Observed outcomes Insufficient history Popularity or content baseline DATA Behavior events + catalog exposures ≠ outcomes MODEL Offline training + index refresh offline metrics require online validation EXTERNAL User or anonymous request MODEL Candidate generation MODEL Ranking model DETERMINISTIC Eligibility + diversity rules EXTERNAL Recommendations DETERMINISTIC FALLBACK Cold-start fallback still applies eligibility + privacy DATA Exposure + outcome logging click and conversion remain distinct Flow legend Primary data path Control Feedback Cold-start branch Popularity bias remains a risk. Offline quality does not by itself demonstrate business lift.

Main walkthrough, step 1 of 6: Prepare models and indexes. idle.

Main walkthrough

The primary path through this reference architecture.

Active parts: Behavior events + catalog, Offline training + index refresh, Candidate generation, Ranking model

Text walkthrough

This vertical reference remains readable without the interactive controls.

  1. 01Prepare models and indexes

    Offline interactions prepare separately approved retrieval assets and a ranking model.

    Active parts: Behavior events + catalog, Offline training + index refresh, Candidate generation, Ranking model

  2. 02Receive a request

    The online path receives current context with its authorization boundary.

    Active parts: User or anonymous request, Candidate generation

  3. 03Retrieve candidates

    Candidate generation narrows the catalog before richer scoring.

    Active parts: Candidate generation, Ranking model

  4. 04Rank and constrain

    The ranking model orders candidates, and deterministic rules produce the eligible displayed set.

    Active parts: Ranking model, Eligibility + diversity rules, Recommendations

  5. 05Record exposures and outcomes

    The displayed order and its request context are recorded before later outcomes are joined.

    Active parts: Recommendations, Exposure + outcome logging

  6. 06Feed future learning

    Observed exposure and outcome events become versioned evidence for a later offline run.

    Active parts: Exposure + outcome logging, Behavior events + catalog

Connections

  • Historical interactionsBehavior events + catalog to Offline training + index refreshdata
  • Embeddings + indexOffline training + index refresh to Candidate generationcontrol
  • Approved ranking modelOffline training + index refresh to Ranking modelcontrol
  • Authorized contextUser or anonymous request to Candidate generationdata
  • Candidate itemsCandidate generation to Ranking modeldata
  • Ranked candidatesRanking model to Eligibility + diversity rulesdata
  • Eligible diverse resultsEligibility + diversity rules to Recommendationsdata
  • Exposure contextRecommendations to Exposure + outcome loggingfeedback
  • Observed outcomesExposure + outcome logging to Behavior events + catalogfeedback
  • Insufficient historyCandidate generation to Cold-start fallbackcontrol
  • Popularity or content baselineCold-start fallback to Eligibility + diversity rulesdata

Components

Behavior events + catalogdata

Keeps authorized interaction, exposure and catalog data with event-time and availability context.

Responsibilities

  • Separate exposures from later outcomes
  • Version catalog and eligibility attributes

Inputs

  • Displayed-item events
  • Clicks or other outcomes
  • Catalog changes

Outputs

  • Historical interactions and catalog snapshot

Risks

  • Missing exposure logs bias training
  • Behavior data is retained beyond its authorized purpose
Offline training + index refreshmodel

Builds candidate-retrieval assets and an approved ranking model from versioned offline data.

Responsibilities

  • Train and evaluate retrieval and ranking separately
  • Refresh compatible indexes and model artifacts

Inputs

  • Historical interactions
  • Catalog snapshot
  • Versioned training code

Outputs

  • Embeddings and candidate index
  • Approved ranking model

Risks

  • Popularity bias is learned and amplified
  • Model and candidate index versions become incompatible
User or anonymous requestexternal

Provides the current page or task plus only the identity and session context permitted for recommendation.

Responsibilities

  • Establish authorized context
  • Support an anonymous path without fabricated history

Inputs

  • Current request
  • Permitted user or session context

Outputs

  • Authorized recommendation context

Risks

  • Untrusted identifiers impersonate another user
  • Sensitive context is exposed to ranking unnecessarily
Candidate generationmodel

Narrows the catalog to a tractable candidate set using retrieval assets and request context.

Responsibilities

  • Retrieve a broad, relevant candidate pool
  • Expose when history is insufficient

Inputs

  • Authorized recommendation context
  • Embeddings and candidate index

Outputs

  • Candidate items
  • Or cold-start signal

Risks

  • Useful items are omitted before ranking
  • Popular items crowd out long-tail options
Ranking modelmodel

Scores the smaller candidate pool with richer request, item and permitted behavioral features.

Responsibilities

  • Produce comparable scores for the candidate pool
  • Attach model version to the ranking

Inputs

  • Candidate items
  • Authorized ranking context
  • Approved ranking model

Outputs

  • Ranked candidates

Risks

  • The objective rewards clicks over lasting value
  • Position bias contaminates learned preferences
Eligibility + diversity rulesdeterministic

Applies inventory, permission, regional, safety and diversity rules before display.

Responsibilities

  • Remove ineligible items
  • Enforce declared diversity and repetition limits

Inputs

  • Ranked candidates
  • Current business and eligibility rules

Outputs

  • Eligible diverse results

Risks

  • Stale availability exposes unusable items
  • A diversity rule unintentionally removes relevant options
Recommendationsexternal

Displays the eligible ordered set and the context needed to record what was actually shown.

Responsibilities

  • Render the approved result order
  • Emit an exposure event for displayed items

Inputs

  • Eligible diverse results

Outputs

  • Displayed recommendations
  • Exposure context

Risks

  • Rendered order differs from the logged order
  • Unavailable items remain visible from stale caches
Exposure + outcome loggingdata

Records displayed items and positions separately from later clicks, purchases or other outcomes.

Responsibilities

  • Join outcomes to real exposures
  • Preserve request and model versions under retention policy

Inputs

  • Exposure context
  • Observed outcomes

Outputs

  • Observed exposure and outcome events

Risks

  • A missing exposure is treated as a negative outcome
  • Logging creates unnecessary privacy risk
Cold-start fallbackdeterministic

Selects a declared popularity or content baseline when useful behavioral history is unavailable.

Responsibilities

  • Return a transparent non-personalized baseline
  • Preserve eligibility and privacy constraints

Inputs

  • Cold-start signal
  • Current catalog context

Outputs

  • Popularity or content baseline

Risks

  • The baseline entrenches popularity bias
  • A global baseline ignores regional availability

Scenario reference

Cold start

Use an explicit baseline, respecting eligibility and privacy constraints.

  1. 01Receive a request without useful history

    The authorized request reaches candidate generation, which declares insufficient behavioral history.

    Active parts: User or anonymous request, Candidate generation, Cold-start fallback

  2. 02Constrain and serve the baseline

    A popularity or content baseline passes through the same eligibility rules before display.

    Active parts: Cold-start fallback, Eligibility + diversity rules, Recommendations

Design Decisions

  • Optimize candidate generation for useful recall, then rank a smaller pool with richer features.
  • Apply eligibility, policy and diversity after ranking but before display.
  • Log item, position, request and model versions for every actual exposure.
  • Provide a transparent cold-start baseline instead of pretending personalization exists.

Risks & Trade-offs

Popular items can dominate retrieval and reinforce their own visibility. Optimizing clicks can reward clickbait rather than durable value. Sparse or delayed outcomes weaken evaluation, while exposure logging introduces privacy and retention obligations. Offline ranking metrics do not prove user or business benefit; controlled online evaluation may still be needed.

When to Use / Not Use

Use a multi-stage engine when the catalog is large, requests need low latency and ranking benefits from context. Prefer curated lists, search or simple rules for small catalogs, infrequent choices or domains where personalization cannot be justified. Do not personalize without an authorized data purpose and an acceptable fallback.

Sources