PixelRAG and DecisionRAG: diagnose what your RAG pipeline cannot find

PixelRAG and DecisionRAG address two different reasons a retrieval-augmented generation system misses evidence. PixelRAG preserves visual structure by retrieving screenshot tiles. DecisionRAG uses decision models to choose paths through a document hierarchy. Neither makes a weak evaluation set or an incorrect permission boundary disappear.

For an FDE, the useful question is where the evidence stops being usable: during capture, representation, candidate selection or reading. Identify that boundary before replacing a pipeline. This article explains the two methods and gives you a controlled experiment you can run on your own corpus.

Start with one answer-bearing source

Imagine a synthetic support question: “What is the overnight replacement limit for contract B?” The source is a table with contract names across columns, service levels down rows and a footnote restricting eligibility.

Text extraction might retain every number but drop the row-to-column relationship. A search result containing “overnight” and “contract B” can then be relevant without preserving enough information to answer. Conversely, a perfect screenshot is useless if retrieval never selects it.

Freeze the original source and inspect four artifacts: the captured page, the indexed representation, the selected context and the final answer. Our document parsing concept provides the foundation; the evidence debugging lab turns it into a repeatable investigation.

How PixelRAG retrieves visual evidence

The official PixelRAG repository separates rendering, embedding, indexing and serving. Documents become image tiles; a visual embedding model makes them searchable; selected images go to a vision-capable reader. Its pipeline supports a Qwen3-VL embedding model, a screenshot-trained adapter and FAISS indexes. The standalone pixelshot renderer is distinct from running a complete retrieval service.

The figure contrasts two representations of the same source. Both paths still need ranking, permission checks and a reader that can interpret what it receives.

The PixelRAG paper studies representation, retrieval and reader choices separately. It also documents storage overhead and the loss of directly actionable hyperlink structure in screenshots. Those limitations matter when a customer expects an answer to cite a source and navigate to its supporting policy.

Treat the paper's results as experiments under stated configurations. Its reported 36.6% parser-loss share refers to analyzed text-path failures, not to all enterprise queries. The appendix uses a bounded evidence search. A miss within a search depth does not, by itself, prove that evidence is absent from the entire corpus.

Build a small comparison before a large index

Choose a permitted, local collection with several document classes: prose, tables, charts, scanned pages and mixed layouts. Label the exact source region needed for each question. Include questions whose correct answer is that the document does not support the requested conclusion.

Start from the repository's local indexing instructions. Pin a reviewed repository revision and dependencies. Inspect captured tiles before downloading a large public index or launching embedding jobs. A public demonstration endpoint is suitable for public examples, not private customer documents or queries.

Store more than an image and a vector. A useful tile record carries source identity, content hash, capture time, source revision, page or tile coordinates, renderer configuration, embedding revision and access scope. Preserve link targets separately if users need them. Keep permissions current even when the image itself has not changed.

Compare these four conditions on the same authorized source snapshot:

The fixed-evidence conditions remove ranking from that comparison. If both fail, inspect the question, labels and reader before spending time on another index. If only extracted text fails, inspect relationships and qualifiers lost in conversion. If the image condition succeeds with fixed evidence but fails end to end, investigate capture coverage, selection and context assembly.

DecisionRAG changes how the search narrows

DecisionRAG evaluates candidate branches with decision-model outputs, then descends into retained branches. Binary relevance judgments can screen candidates; a final multiple-choice comparison can refine their ordering. It avoids generating reasoning text for each judgment. The method still performs model inference and needs a useful hierarchy.

Its pruning is irreversible within that query: a relevant passage under a discarded parent never reaches the final reader. The paper also permits adding dense-retrieval candidates to recover evidence missed by navigation. This is a useful distinction from “replace every retriever.”

For your own experiment, log the path to each labeled relevant leaf. Record the first level where that path was rejected, the retained budget and the candidate representation shown to the scorer. A vague document title can hide an exception buried in the full document. Increasing the final passage count cannot recover a branch already discarded above it.

Keep alternative paths when your measured recall requirements justify their cost. Evaluate the resulting latency under actual concurrency: parallel scoring still competes for capacity. Our decision-model concept explains typed outputs, while retrieval evaluation explains what to measure after selection.

Preserve security and source meaning

Rendering a page is active work. Run untrusted capture in an isolated environment, limit outbound access, enforce time and resource budgets, and keep customer sessions separate. Check redirect destinations and disallow internal network targets where URLs are user-controlled. An image can contain prompt-injection instructions just as text can; retrieved pixels remain untrusted evidence.

Before returning evidence, enforce tenant and document access through permission-aware RAG. Do not retrieve broadly and rely on the reader to hide forbidden content. Ensure a revoked document disappears from image storage, vectors, caches and derived excerpts according to the service's deletion contract.

A citation should identify the exact capture used. Re-rendering a mutable page after retrieval may produce different evidence. If retaining images is too expensive, retain enough source material and renderer configuration to reproduce the cited capture, or explicitly report that reproducibility is unavailable.

What belongs in the release report

Report evidence recall, answer correctness and citation support separately, sliced by document class. Add capture failures, unreadable tiles, abstentions, storage, ingestion time, total model cost and request latency. Keep failed requests in the accounting. A chart slice can improve while prose quality or capture reliability deteriorates.

Use a held-out set to confirm the selected configuration. Synthetic questions help test specific boundaries but do not estimate traffic quality unless their sampling matches that traffic. Compare quality at a stated cost and latency budget, rather than declaring a win from a single overall accuracy score.

Then try the visual evidence diagnostic exercise. It supplies a small failure packet and asks you to choose the repair from observed evidence. For the broader architecture, continue to multimodal retrieval over slide decks.