EmbeddingGemma 2: Build, Fine-Tune and Deploy Multimodal Search

EmbeddingGemma 2 turns text, code, images, audio and video into vectors that an application can compare. Use it to find the right runbook, locate a function by behavior, search a recording for a relevant segment, or suggest a support category. Start with text retrieval, measure it on your own questions, and add modalities only when the task needs them. Google released the model on October 6, 2026; this tutorial checks the published interfaces as of October 28. Google's developer guide introduces the release and its modular encoders.

The most useful first project is small: a search service that returns evidence a user is allowed to see, together with the source location. You can add answer generation later. The companion DiffusionGemma tutorial covers that separate stage, including coding and decision-support workflows.

Understand what the model gives you

An embedding is a list of numbers representing an input. Related inputs should produce nearby vectors. Searching means encoding a query, comparing it against stored document vectors, and returning the best candidates. The embeddings lesson explains the geometry without assuming a machine-learning background.

Suppose a technician asks, “Where does the manual show the pressure-release valve?” The relevant evidence might be a diagram, while the query is text. A shared multimodal space makes that comparison possible. The application must still preserve the diagram's page number and equipment version so the technician can check the original.

The checkpoint is google/embeddinggemma-2. Its published configuration supports 768-dimensional embeddings and an 8,192-token input window. Smaller supported representations use the first 512, 256 or 128 dimensions, followed by rescaling each vector to unit length. Google distributes it under Apache 2.0. These are properties of this checkpoint, not a promise that every runtime exposes every modality. Official model overview.

The architecture is modular:

The numbers describe weights, not total process memory. Tokenization, intermediate activations, batching and the vector index also consume memory. The same checkpoint's supported encoder configurations share an embedding space; an independently fine-tuned model does not inherit that compatibility automatically. Google's configuration guidance.

In the diagram, every input reaches the same representation stage, but the source remains separate. Keep source identifiers outside the vector: the vector helps locate an item; it cannot reconstruct the original manual or establish who may read it.

Choose a useful first application

These are proposed engineering projects, not claims that the model has passed the named workload. Each has a concrete output and a check that can expose failure.

For an FDE, the first row is usually the easiest to evaluate with a customer. Ask them for real questions and the passages they would trust. A narrow collection with reviewed answers is more informative than indexing a whole drive and judging five attractive results.

Use an ordinary exact lookup when someone supplies an invoice number, repository symbol or error code. Add hybrid search when queries mix identifiers and descriptions. A semantic model can connect “replay after a lost response” with “retry policy,” but it should not decide that ERR_4071 and ERR_4077 are close enough.

Run a small text-retrieval example

Create a separate Python environment. The reference below uses sentence-transformers 6.2.0 and transformers 5.21.0; Google's release guide specifies Sentence Transformers 6.1.0 or later. Use a PyTorch build appropriate to your machine. This example explicitly selects CPU and float32. The multimodal processor still imports media dependencies even when the unused model encoders are disabled, so install the media packages below. Choose the CPU PyTorch packages if you do not need CUDA.

python3 -m venv .venv
source .venv/bin/activate
python -m pip install "sentence-transformers==6.2.0" "transformers==5.21.0" torchvision pillow soundfile

Save the following as search_demo.py. The three records are synthetic. There is deliberately one record belonging to another tenant. The model revision is pinned so a later checkpoint update does not silently change the example.

"""Small synthetic, permission-scoped retrieval example; not a quality benchmark."""
import numpy as np
import torch
from sentence_transformers import SentenceTransformer

MODEL_ID = "google/embeddinggemma-2"
REVISION = "f9972ac22e696a105c482202646979e33e4b5a1e"
model = SentenceTransformer(
    MODEL_ID, revision=REVISION,
    config_kwargs={"vision_config": None, "audio_config": None},
    model_kwargs={"dtype": torch.float32}, device="cpu",
)
documents = [
    {"id": "retry", "tenant": "demo", "text":
     "After a timeout, look up the original operation ID before retrying a write."},
    {"id": "reindex", "tenant": "demo", "text":
     "Build a separate index when changing embedding models; switch after evaluation."},
    {"id": "private", "tenant": "other", "text":
     "A different customer's private timeout incident. Do not disclose it."},
]

def embed(texts, prompt):
    vectors = model.encode(texts, prompt_name=prompt, truncate_dim=256,
                           normalize_embeddings=True)
    vectors = np.asarray(vectors, dtype=np.float32)
    if not np.isfinite(vectors).all():
        raise ValueError("Non-finite embedding")
    if not np.allclose(np.linalg.norm(vectors, axis=-1), 1.0, atol=0.02):
        raise ValueError("Embedding is not normalized")
    return vectors

def search(query, tenant, k=2):
    # Production tenant identity must come from authenticated server context.
    allowed = [d for d in documents if d["tenant"] == tenant]
    if not allowed:
        return []
    corpus = embed([d["text"] for d in allowed], "Document")
    query_vector = embed([query], "SearchQuery")[0]
    scores = corpus @ query_vector
    order = np.argsort(-scores, kind="stable")[:k]
    return [(allowed[i]["id"], float(scores[i])) for i in order]

if __name__ == "__main__":
    print(search("The request timed out. Is it safe to send it again?", "demo"))
    assert search("timeout", "missing-tenant") == []
    assert all(doc != "private" for doc, _ in search("private timeout", "demo"))

python search_demo.py

The output is a ranked list of document IDs and similarity scores. Read the returned text to check the ranking. The assertions establish two access-filtering behaviors in this tiny example; they do not establish retrieval accuracy on your collection. Scores vary with runtime and numerical precision, so do not use a copied decimal as a production threshold.

The example recomputes document embeddings on each query to make the complete operation visible. In a service, compute them during ingestion, save them with source metadata, and encode only the incoming query. Start with exact matrix multiplication for a small collection. Adopt an approximate nearest-neighbor index when measurements show that exact search is too expensive.

The accompanying text example, image example and API were executed on CPU. The training script was exercised for two optimizer steps with synthetic records, then its saved model was reloaded. These checks verify the code paths; they are not a retrieval-quality benchmark or evidence that two training steps improve a model.

The two task prompts matter. SearchQuery identifies the text as a search request; Document formats it as searchable material. For a document with a real title, Google's convention is title: {title} | text: {content}. If you construct that string yourself, use an empty prompt instead of adding Document a second time. The official text tutorial documents these interfaces.

Use bfloat16 or float32, not float16, for this model. Its activation range can produce non-finite or degraded results in float16. Check both finiteness and vector norms, including after a precision or runtime change. On compatible accelerators, bfloat16 is a reasonable candidate; benchmark it against the float32 reference before changing a production index. Numerical-precision guidance.

Build the index around permissions and evidence

A production record needs more than an embedding. Store a stable source ID, content hash, tenant or access scope, source version, chunk location and preprocessing version. For media, include a page number or start/end timestamps. Store the model revision and dimension in the index manifest.

The query path in this diagram applies authorization before selecting results. Filtering the top ten afterward can leave a user with no results even when relevant authorized items exist deeper in the index. More seriously, fetching unauthorized text into a reranker or generator already crosses the data boundary, even if the UI later hides it.

In a shared vector database, use an enforced tenant partition or a correctly implemented metadata filter. Derive that scope from authenticated server context. A request body's tenant field is not proof of identity. Recheck source permissions when returning the original content, especially if permissions can change faster than the indexing job.

For a RAG application, return evidence identifiers alongside text and have the generator cite those identifiers. Retrieve the minimum necessary context. Treat retrieved documents as data, including when they contain instructions aimed at the model. The free retrieval troubleshooting lesson walks through locating the failure before changing a prompt.

Select chunks that a person can verify

A large input window lets more content fit; it does not make an entire document the right retrieval unit. A fifty-page manual compressed into one vector may rank as generally relevant while hiding the only paragraph that answers the question.

Split by meaningful boundaries. A runbook step should retain its heading and conditions. A code function should retain its signature and relevant type information. A slide should retain its page number. Audio chunks should be short enough that listening to the returned segment is practical. Overlap can preserve context, but duplicate chunks should not occupy all of the top results.

Evaluate the representation at the level where the user needs evidence. If a task requires exact values from a table, combine visual retrieval with text extraction and a numerical check. Similarity to a chart is insufficient to answer what a particular bar measures.

Extend the same application to images, audio and video

Install the media dependencies from the multimodal tutorial. Video decoding also needs a compatible TorchCodec/FFmpeg environment; test decoding separately before diagnosing the model.

python -m pip install pillow soundfile torchcodec

For text-to-image retrieval, load the vision encoder and provide local image files through Pillow. Save your own two sample images as manual-page.png and control-panel.png. This example uses the same pinned checkpoint as the text example.

"""Encode two user-provided images and a text query on CPU."""
from PIL import Image
from sentence_transformers import SentenceTransformer

MODEL_ID = "google/embeddinggemma-2"
REVISION = "f9972ac22e696a105c482202646979e33e4b5a1e"
vision = SentenceTransformer(
    MODEL_ID, revision=REVISION, device="cpu",
    config_kwargs={"audio_config": None},
    model_kwargs={"dtype": "float32"},
)
pages = [Image.open(p).convert("RGB")
         for p in ["manual-page.png", "control-panel.png"]]
page_vectors = vision.encode(pages, normalize_embeddings=True)
query_vector = vision.encode(
    ["Where is the emergency stop control?"],
    prompt_name="SearchQuery", normalize_embeddings=True,
)
print(vision.similarity(query_vector, page_vectors))

A returned matrix contains one query-to-image score per image. It is a retrieval result, not a location annotation: the model has not drawn a box around the emergency stop. Present the page and let the user inspect it, or add a separately evaluated localization step.

The multimodal API also accepts modality-keyed inputs such as {"audio": "clip.wav"} and {"video": "clip.mp4"}. Load the audio encoder for audio. Supply mono, 16 kHz audio and keep sampling settings in your preprocessing manifest. Video processing samples frames; a frame-based representation can miss a brief event between samples. If speech matters, process the audio track separately and join results by timestamp. Official multimodal input examples.

For a training-video library, store several short segments rather than only one vector for an hour-long recording. A search for “how to reset the controller” should open near the relevant action. Review whether the reset is actually visible, spoken about, or merely mentioned in a slide. Those are different labels in an evaluation set.

For mixed text and media, the official processor supports interleaved inputs. Preserve the order and association between the caption and the media. Combining an image with its product metadata may improve retrieval, but a stale caption can dominate the result. Keep a visual-only evaluation slice so you can detect that failure.

Use it in a coding assistant

Index source code at function or class boundaries. Save repository, commit, path, symbol and line range in metadata. Exclude secrets, generated dependencies and build outputs before indexing. A question such as “Where do we stop duplicate webhook processing?” should return the implementation and relevant tests, not every file containing “webhook.”

The checkpoint's CodeRetrieval prompt is intended for code-retrieval queries. Keep document formatting consistent and compare it with SearchQuery on your actual query set; do not switch prompts without measuring. The published checkpoint prompt configuration is the precise reference for names and prefixes.

A useful coding workflow has two retrieval stages. First, exact symbol/path search finds explicit identifiers. Semantic retrieval finds behaviorally related code. Merge candidates, remove duplicates and inspect dependencies before proposing a patch. A candidate function from an obsolete branch should never become the basis for editing current main.

After a branch switch, invalidate changed content and deleted files. Keep the old index available until the new generation is complete. The versioned-index course lesson teaches the publication boundary, including changes to permissions and parsing. That mechanism is more valuable than continually increasing the number of chunks returned to the model.

Use similarity for decision support carefully

A support router can encode an incoming ticket and compare it with reviewed category descriptions. Another approach compares it against labeled examples and aggregates their labels. Both create a candidate decision. Neither establishes a calibrated probability of correctness.

Start with a review-only workflow: show the proposed queue, the nearest examples and a “none of these” option. Collect corrections. Separate a low top score from a small margin between the top two categories; they expose different ambiguities. Tune thresholds on held-out data and report the error rate among the cases that would actually be routed automatically.

Do not adopt a universal “cosine above 0.8 means safe” rule. A model update, different prompt, new language or truncated vector can move the score distribution. Even a softmax over category scores remains an uncalibrated transformation unless you validate it against outcomes. The calibration concept explains why ranking and reliable probabilities are separate tasks.

Useful low-consequence applications include suggesting a documentation label, grouping similar incident reports, or recommending which evidence an operator should inspect next. If a workflow can approve a refund, change permissions or suppress an urgent alert, deterministic policy and human authority need their own controls. The model does not inherit permission from a semantically similar past case.

Evaluate before you fine-tune

Create a review sheet with query, permitted corpus, relevant IDs, expected evidence and failure category. Include exact identifiers, internal abbreviations, multilingual queries, outdated documents and questions with no supported answer. Split by document family, incident or customer where appropriate, so paraphrases of the same record do not leak into both training and test sets.

Compare the same collection under these baselines: lexical retrieval; the untuned embedder; hybrid retrieval; and, if needed, a reranker. The internal-jargon question explains why a glossary or lexical path can solve a problem more cheaply than training.

Use Hit@k when the question is whether at least one useful result appears in the first k. Use Recall@k when multiple relevant items exist and you care how many are recovered. For example, retrieving one of four relevant passages in the first five gives Hit@5 = 1 and Recall@5 = 0.25. Neither says whether the generator's answer was correct. Measure answer support separately.

Also record query latency percentiles, ingest throughput, peak memory, empty-result behavior and forbidden-result count. Break quality down by language and modality. An average can conceal a severe regression for short audio clips or visually dense pages.

Reduce vector storage with a measured dimension choice

For one million float32 vectors, the raw vector payload is count × dimensions × 4 bytes:

These are arithmetic estimates, excluding index structures, metadata, replicas and backups. Reducing dimensions reduces this payload; it does not make the entire database six times smaller. Normalize after truncation and use the same dimension for both sides of every comparison. Google's dimension evaluation reports different quality changes across modalities, so test 128 dimensions especially carefully for media.

Fine-tune on failures you can explain

Fine-tuning adjusts which examples the model places near one another. Prepare a query, a relevant item and a confusing but irrelevant item. A useful hard negative for “retry a timed-out write” is a procedure for retrying a read, not an unrelated recipe. Review negatives: two passages may both be valid, and training one as wrong teaches the model an artificial distinction.

Google provides separate text tuning and multimodal tuning examples. The text path uses contrastive training through Sentence Transformers. Preserve the query/document prompt conventions used by your application rather than copying a tutorial's symmetric similarity setup into an asymmetric retrieval task.

Create train.jsonl containing reviewed records with exactly anchor, positive and negative string fields. For example:

{"anchor":"A write timed out. Can I retry?","positive":"Check the original operation receipt before repeating a write.","negative":"A read-only health check can be retried with bounded backoff."}

One record is a format example, not sufficient training data. Begin with a small reviewed collection that covers several failure families and preserve a separate validation set. The following training skeleton keeps one epoch as a starting configuration, not a quality recommendation.

"""Text retrieval training skeleton. Supply reviewed train.jsonl; evaluate separately."""
import os
from datasets import load_dataset
from sentence_transformers import (
    SentenceTransformer, SentenceTransformerTrainer,
    SentenceTransformerTrainingArguments,
)
from sentence_transformers.sentence_transformer.losses import MultipleNegativesRankingLoss
from sentence_transformers.sentence_transformer.training_args import BatchSamplers

model = SentenceTransformer(
    "google/embeddinggemma-2",
    revision="f9972ac22e696a105c482202646979e33e4b5a1e",
    config_kwargs={"vision_config": None, "audio_config": None},
    model_kwargs={"dtype": "float32"},
)
model.max_seq_length = 256
train = load_dataset("json", data_files="train.jsonl", split="train")
train = train.select_columns(["anchor", "positive", "negative"])
args = SentenceTransformerTrainingArguments(
    output_dir="embeddinggemma-domain-run",
    num_train_epochs=1, per_device_train_batch_size=2,
    learning_rate=2e-5, warmup_steps=0.1,
    fp16=False, bf16=False,
    batch_sampler=BatchSamplers.NO_DUPLICATES,
    prompts={"anchor": model.prompts["SearchQuery"],
             "positive": model.prompts["Document"],
             "negative": model.prompts["Document"]},
    report_to="none", save_strategy="no",
    max_steps=int(os.environ.get("TRAIN_MAX_STEPS", "-1")),
)
trainer = SentenceTransformerTrainer(
    model=model, args=args, train_dataset=train,
    loss=MultipleNegativesRankingLoss(model),
)
trainer.train()
model.save_pretrained("embeddinggemma-domain-candidate")

Install datasets and accelerate, save the script as train_embeddings.py, then run python train_embeddings.py beside your train.jsonl. The 256-token training limit is a deliberate starting budget: inspect and chunk examples before training so relevant text is not silently cut off. The default uses float32; select a supported precision only after checking memory and numerical behavior.

Run the resulting model against the frozen validation queries before considering a test-set evaluation. A lower training loss is weak evidence if retrieval got worse on ordinary requests. With in-batch negatives, remove duplicate and semantically equivalent positives from the same batch; the no-duplicates sampler catches repeated values, not every semantic false negative.

If reduced dimensions matter in production, evaluate them after training too. Ordinary contrastive tuning does not guarantee that the original small-vector behavior survives. Sentence Transformers also provides Matryoshka training losses; use an objective and evaluation that match the dimensions you intend to serve.

For cross-modal tuning, construct pairs that express the actual task, such as a technician's text request and the correct diagram. Keep task prefixes on text inputs and preserve image/audio preprocessing. Hold out equipment families or recording sessions rather than nearby frames from the same video. Retest text retrieval as well: changing shared weights can improve one modality and damage another.

Host it as a small service

For a single process, loading the model once and calling it directly is simplest. A service boundary becomes useful when several applications share a model, you need independent resource limits, or ingestion and querying have different schedules.

Save this as embedding_api.py beside search_demo.py. It deliberately exposes only text embeddings, limits batch size and token length, and rejects concurrent work instead of allowing an unbounded GPU or CPU queue. It is a loopback teaching service; keep it private until your deployment supplies authentication and request-body limits.

"""Loopback teaching API; authentication belongs at a private production ingress."""
from threading import BoundedSemaphore
from typing import Annotated, Literal
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field
from search_demo import MODEL_ID, REVISION, embed, model

app = FastAPI()
slots = BoundedSemaphore(1)

class Batch(BaseModel):
    texts: list[Annotated[str, Field(min_length=1, max_length=12000)]] = Field(
        min_length=1, max_length=16)
    kind: Literal["query", "document"] = "query"

@app.post("/embed")
def encode(batch: Batch):
    prompt = "SearchQuery" if batch.kind == "query" else "Document"
    for text in batch.texts:
        ids = model.tokenizer.encode(model.prompts[prompt] + text)
        if len(ids) > 512:
            raise HTTPException(413, "Split inputs to at most 512 tokens including prefix")
    if not slots.acquire(blocking=False):
        raise HTTPException(429, "Encoder busy; retry with backoff")
    try:
        vectors = embed(batch.texts, prompt)
        return {"model": MODEL_ID, "revision": REVISION,
                "dimensions": 256, "vectors": vectors.tolist()}
    finally:
        slots.release()

python -m pip install fastapi uvicorn
python -m uvicorn embedding_api:app --host 127.0.0.1 --port 8081 --workers 1

In a second terminal:

curl --fail http://127.0.0.1:8081/embed \
  -H 'Content-Type: application/json' \
  -d '{"texts":["How do I recover a timed-out write?"],"kind":"query"}'

The response includes the exact revision and dimension, allowing the caller to reject incompatible vectors. A production service also needs authenticated clients, timeouts, bounded request bytes at ingress, metrics and model-readiness checks. Do not log raw customer documents by default. Download approved weights during packaging or controlled startup; do not give every request the ability to choose an arbitrary model or fetch a URL.

One worker avoids loading a model copy per web process. For higher throughput, measure batching with representative text lengths. A busy endpoint should apply backpressure; retrying every failed batch immediately can turn a recoverable backlog into an outage. A deployment with a GPU also needs limits on concurrent media decoding and accelerator memory.

AWS, Google Cloud and Azure can host an appropriate container or GPU workload; the deployment decision is about hardware and operations, not a requirement to change the embedding model. Avoid publishing a universal hourly price. Compare cost per accepted ingestion item or per successful query, including idle capacity and the people maintaining the service. The Hard Deployments implementation lesson develops that service lifecycle further.

Migrate indexes without mixing vector spaces

Changing from EmbeddingGemma 1, Gemini Embedding or another embedding model requires a new index. Equal dimensions only mean equal array lengths. They do not establish shared coordinates. Fine-tuning also changes the space.

Build the candidate index beside the current one. Replay the same queries against both, keeping permissions and source versions aligned. Switch the query encoder and active index together through a manifest or versioned configuration. Preserve a rollback route to the old pair until the candidate passes live checks. A partial re-embedding job must not quietly become the active corpus.

Deletion and permission revocation need their own tests. Remove vectors, cached retrieval results and copied source text according to the application's retention rules. A local model can reduce external data transfer while the surrounding logs or media URLs still send data elsewhere. “Runs locally” is a property to verify across the whole request path.

Public projects worth reading

The following repositories are implementation references, not endorsements or independent benchmark results. Inspect their current code and supported hardware before adopting them.

• Latent's local embedding integration documents a separate EmbeddingGemma 2 index and an explicit switch from its existing Gemini-backed search. The useful lesson is preserving the old engine and its data during migration.
• embeddinggemma.c explores a specialized native inference server, including batching and bounded queues. Check backend support for the exact model profile; EmbeddingGemma 300M support does not establish EmbeddingGemma 2 support on the same accelerator.
• Transformers' EmbeddingGemma2 implementation exposes the preprocessing, pooling and normalization path. Use it as a reference when checking whether an optimized runtime produces compatible vectors.

Read public examples for their boundaries as carefully as their demos: supported input type, model revision, missing capabilities and how results were measured. A screenshot of a good search result is a starting hypothesis.

Build a first deliverable a customer can review

Choose a permitted collection, record its version, and label a set of real questions before tuning anything. Deliver a search screen that opens the exact source, a comparison with lexical retrieval, and a report of failures by category. Add one meaningful alternative, such as hybrid search or a smaller vector dimension, and explain the observed tradeoff.

The next exercise is to change the corpus: revise a procedure, delete a document, revoke a permission and switch the model revision. Check that the system returns current, permitted evidence or clearly reports that it cannot answer. That is the point where an embedding demo becomes useful deployment work.