Adaptation & Evaluation

Prompting, RAG or Fine-Tuning?

Compare prompt-time instructions, retrieval of current evidence and weight adaptation against one shared evaluation goal.

Reference architecture - illustrative, not a client deployment

Compare runtime context with weight updates, including a combined approach.

Complexity
Intermediate
Drawing
Supplied
  • Prompting
  • RAG
  • Fine-tuning
  • Evaluation

Overview

This reference architecture is illustrative. It is not a record of a client deployment.

Prompting, retrieval-augmented generation (RAG) and fine-tuning change different parts of an LLM system. Prompting supplies instructions and examples at request time while leaving the base model unchanged. RAG also leaves weights unchanged: it retrieves current, authorized evidence and adds it to the request context. Fine-tuning trains model parameters or adapters from authorized examples to shape behavior. The three approaches can be combined; they are not a maturity ladder.

Start with a task definition and a held-out evaluation set. A prompt baseline establishes whether the unchanged model already has the capability. Add retrieval when the answer depends on changing or source-specific knowledge. Consider fine-tuning when repeated prompting cannot reliably establish the desired format or behavior. Compare every candidate on the same held-out cases rather than changing the test when the implementation changes.

A concrete example

A support assistant must answer from a policy that changes quarterly and always return a compact escalation category. Retrieval supplies the current policy passage. A fine-tuned adapter may improve the consistency of the category and response structure. The combined system still needs the retrieved evidence; encoding last quarter's policy into weights would not make it current.

Common misconception

"Fine-tuning uploads the latest facts into a dependable database." Training can influence both knowledge and behavior, but weights are not a traceable, permission-aware source of frequently changing truth. Use retrieval when freshness, citations or access control matter.

Diagram & Walkthrough

Choose a declared scenario and step, or select a component by pointer, touch or keyboard. Playback starts only when you press Play and pauses when this panel leaves the screen.

100%
Prompting, RAG or Fine-Tuning? A comparison of prompt-time context, retrieval of changing authorized evidence, weight updates through fine-tuning, and a combined retrieval plus adapted-model approach, all measured by one held-out evaluation. Prompting, RAG or Fine-Tuning? Runtime context and weight updates solve different problems—and can be combined. GOAL AND SHARED EVALUATION PROMPTING · RUNTIME CONTEXT weights remain unchanged RETRIEVAL · FRESH EVIDENCE AT RUNTIME TRAINING · WEIGHT OR ADAPTER UPDATE Define the task Prompt-time context Fresh authorized evidence Retrieved context Target behavior Train split Updated weights or adapters Task instructions RAG + adapted model Prompt or RAG outputs Adapted outputs HUMAN Task + evaluation goal DETERMINISTIC · SHARED Held-out evaluation same task, same criteria DATA · RUNTIME Prompt + examples instructions and demonstrations MODEL Unchanged base model no weight update DATA Current documents changing external evidence DETERMINISTIC Authorized retrieval access and provenance enforced DATA · TRAINING Authorized training examples separate from held-out evaluation MODEL TRAINING Fine-tuning job behavior and/or knowledge MODEL Adapted model versioned candidate Comparison guide Runtime data path Training data Control / weight update Changing knowledge: begin with retrieval for current, attributable evidence. Consistent behavior: compare an adapted candidate against the prompt-only baseline. Combined: retrieval and fine-tuning are not mutually exclusive. No method is a universal decision rule; evaluate on the task.

Main walkthrough, step 1 of 6: Define the task and tests. idle.

Main walkthrough

The primary path through this reference architecture.

Active parts: Task + evaluation goal, Prompt + examples, Authorized training examples

Text walkthrough

This vertical reference remains readable without the interactive controls.

  1. 01Define the task and tests

    Set one goal and held-out comparison before preparing prompts or training examples.

    Active parts: Task + evaluation goal, Prompt + examples, Authorized training examples

  2. 02Establish a prompt baseline

    Run the unchanged base model from request-time instructions and score its output.

    Active parts: Prompt + examples, Unchanged base model, Shared held-out evaluation

  3. 03Add current evidence through retrieval

    Retrieve permitted current documents, add them to the base-model context and evaluate the result.

    Active parts: Current documents, Authorized retrieval, Unchanged base model, Shared held-out evaluation

  4. 04Adapt behavior through training

    Train on authorized examples and run the adapted candidate with the same task instructions.

    Active parts: Authorized training examples, Fine-tuning job, Adapted model, Prompt + examples

  5. 05Combine approaches when needed

    Send current retrieved evidence to the adapted model when the task needs both fresh knowledge and learned behavior.

    Active parts: Authorized retrieval, Adapted model

  6. 06Compare on the same held-out tests

    Compare baseline, retrieval and adapted outputs without changing the acceptance set.

    Active parts: Unchanged base model, Adapted model, Shared held-out evaluation

Connections

  • Define the taskTask + evaluation goal to Prompt + examplesdata
  • Prompt-time contextPrompt + examples to Unchanged base modeldata
  • Fresh authorized evidenceCurrent documents to Authorized retrievaldata
  • Retrieved contextAuthorized retrieval to Unchanged base modeldata
  • Target behaviorTask + evaluation goal to Authorized training examplescontrol
  • Train splitAuthorized training examples to Fine-tuning jobdata
  • Updated weights or adaptersFine-tuning job to Adapted modelcontrol
  • Task instructionsPrompt + examples to Adapted modeldata
  • RAG + adapted modelAuthorized retrieval to Adapted modeldata
  • Prompt or RAG outputsUnchanged base model to Shared held-out evaluationdata
  • Adapted outputsAdapted model to Shared held-out evaluationdata

Components

Task + evaluation goalhuman

Defines the user task, acceptance criteria and one held-out comparison set before a method is selected.

Responsibilities

  • State required behavior, evidence needs and release criteria

Inputs

  • Business objective
  • Representative failure modes

Outputs

  • Task definition
  • Held-out evaluation goal

Risks

  • Vague success criteria
  • Changing tests to favor a candidate
Prompt + examplesdata

Holds request-time instructions and optional few-shot examples without modifying model weights.

Responsibilities

  • Express the task and response contract within the context window

Inputs

  • Task definition
  • Approved examples

Outputs

  • Prompt-time context

Risks

  • Prompt brittleness
  • Context-window pressure
Unchanged base modelmodel

Produces the prompt-only or retrieval-grounded baseline with its pretrained weights unchanged.

Responsibilities

  • Generate outputs from instructions and optional retrieved context

Inputs

  • Prompt
  • Optional retrieved evidence

Outputs

  • Baseline outputs

Risks

  • Unsupported claims
  • Inconsistent task behavior
Current documentsdata

Stores current source material together with provenance and access-control metadata.

Responsibilities

  • Provide governed, updateable evidence

Inputs

  • Authorized source content
  • Permissions and versions

Outputs

  • Searchable current documents

Risks

  • Stale content
  • Missing or incorrect permissions
Authorized retrievaldeterministic

Selects relevant evidence while enforcing the caller's authorized document scope.

Responsibilities

  • Filter, retrieve and preserve source references

Inputs

  • Question
  • Trusted identity
  • Current documents

Outputs

  • Authorized retrieved context

Risks

  • Access-control bypass
  • Low recall or irrelevant context
Authorized training examplesdata

Contains reviewed examples that may be used to adapt repeated model behavior.

Responsibilities

  • Preserve consent, provenance and split membership

Inputs

  • Approved input-output examples

Outputs

  • Versioned training split

Risks

  • Sensitive-data leakage
  • Test contamination
Fine-tuning jobmodel

Optimizes selected parameters or adapters using the authorized training split.

Responsibilities

  • Run a reproducible adaptation configuration

Inputs

  • Base revision
  • Training examples
  • Training configuration

Outputs

  • Updated weights or adapters

Risks

  • Overfitting
  • Behavioral regressions
Adapted modelmodel

Combines a compatible base with trained weights or adapters and still accepts prompt or retrieval context.

Responsibilities

  • Generate candidate outputs under a versioned serving configuration

Inputs

  • Adapted parameters
  • Task instructions
  • Optional retrieved evidence

Outputs

  • Adapted outputs

Risks

  • Base-adapter incompatibility
  • Unexpected capability loss
Shared held-out evaluationdeterministic

Compares all candidates on the same unseen cases and separately reported criteria.

Responsibilities

  • Apply common quality and safety gates

Inputs

  • Baseline outputs
  • Adapted outputs
  • Held-out cases

Outputs

  • Comparable evaluation report

Risks

  • Biased test coverage
  • Overreliance on aggregate scores

Scenario reference

Changing knowledge

Prefer retrieval as a starting pattern for frequently changing external evidence.

  1. 01Prompt with current evidence

    Frame the task, retrieve fresh permitted evidence and evaluate the unchanged model.

    Active parts: Task + evaluation goal, Prompt + examples, Current documents, Authorized retrieval, Unchanged base model, Shared held-out evaluation

Consistent task behavior

Compare a fine-tuned candidate against a prompt-only baseline.

  1. 01Train and evaluate behavior

    Prepare authorized examples, produce an adapted model and evaluate its instructed output.

    Active parts: Task + evaluation goal, Authorized training examples, Fine-tuning job, Prompt + examples, Adapted model, Shared held-out evaluation

Knowledge and behavior

Retrieval and fine-tuning can be used together.

  1. 01Combine evidence and adaptation

    Train the adapter, supply current retrieved evidence and evaluate the combined candidate.

    Active parts: Current documents, Authorized retrieval, Authorized training examples, Fine-tuning job, Prompt + examples, Adapted model, Shared held-out evaluation

Design Decisions

  • Define measurable quality and safety criteria before selecting an adaptation method.
  • Keep authorized documents and training examples under separate provenance and access controls.
  • Test prompt-only, RAG, adapted and combined candidates against the same cases.
  • Record base-model, prompt, retrieval and adapter versions so results are reproducible.

Risks & Trade-offs

Long prompts consume context and can be brittle. Retrieval adds indexing, authorization and latency concerns. Fine-tuning adds data curation, training, evaluation and serving compatibility work. Any option can regress when the base model or surrounding system changes, and none guarantees correct output.

When to Use / Not Use

Use prompting for a fast baseline, RAG for current governed evidence, and fine-tuning for recurring behavioral adaptation that survives evaluation. Do not fine-tune merely to avoid maintaining a source of changing facts. Do not add retrieval when a deterministic lookup is safer, or train when prompt-only performance already meets the acceptance criteria.

Sources

Related