ADR 0129 — RAG search telemetry is a TABLE, with the weights frozen in the row

Context

apps/api/src/domain/rag/rag-search-limits.ts declares, in its own comment, that the four numbers of the hybrid search — the two weights (0.6/0.4), the threshold (0.2) and the number of candidates per signal — come from no calibration at all:

NENHUM dos quatro números abaixo vem de calibração com dado real: não há, ainda, um corpo de perguntas reais rodado contra este índice.

ADR 0080 chose those values with the reasoning recorded (the scales of ts_rank and cosine are not comparable, and the larger weight on the vector side compensates for the shorter ruler of the lexical one) and said they would be revisited with data. The data never came, and the reason was not lack of use: the search left no trace at all. Zero table rows, zero rag.* events in the whole product. Without a trace, "calibrating" would be swapping one guess for another — and worse, it would be irreversible in practice, because there would be nothing to compare before and after.

This is the first of the five stages of the measurable RAG program, and it is deliberately the only one that does not change search behavior: it gives the eyes, and stops.

The concrete obstacle

The obvious way out would be a session event — the event log is already the measuring instrument for the rest of the product, and the CLAUDE.md rule says execution metrics are extracted from it by script, never by hand. But session_events.session_id is NOT NULL (apps/api/src/db/schema/sessions.ts), and a search coming from the RAG tab is a PROJECT one: it has no session at all. Recording the telemetry only as an event would lose exactly the searches in which a human looked at the scores and judged — which are the ones that carry the only real signal the measurement can have.

It is the same class of problem that forced the cut of embedding metering in ADR 0075: token_usage.session_id is also NOT NULL, and indexing a repository does not happen inside a session. There the cut was declared and the spend was left out. Here the cut would be worse: half the searches would remain, and the ones that remain are the agents' — the ones nobody judges.

Decision

1. Two tables in apps/api/src/db/schema/rag.ts, not events.

rag_searches (one row per search: hits with the 1-based rank of each chunk, degraded/vector_available, latency_ms, actor, and a NULLABLE session_id) and rag_feedback (the util/irrelevante vote on a chunk of a search). The aggregate's file already existed (chunks), so there is no change in the db/schema.ts barrel; the rag_verdict enum lives in the same file as the table that calls it, by the convention of ADR 0121, and actor_kind is reused from sessions.ts instead of redeclared.

session_id NULL is information, not absence of data: it means the search came from the tab.

2. The weights go FROZEN in the row.

pesosVigentes() (domain/rag/rag-telemetry.ts) copies {vector, lexical, threshold} from the moment of the search into the record. It is the same discipline as the price frozen in metering (ADR 0042) and the image_version of project_containers (ADR 0081).

Without the copy, the first calibration that touched RAG_SEARCH_WEIGHT_VECTOR would make all previous measurement start meaning something else, silently — and "did it improve after the change?", the only question the telemetry exists to answer, would become impossible to ask with the data at hand. It is a function and not an exported constant on purpose: a shared constant would be the same object in every JSONB row in memory.

3. The event exists, and it is NARRATION.

rag.search and rag.feedback enter the event log only when there is a session — the agent's path —, so the timeline can narrate what it did. The asymmetry is declared instead of hidden: the table is the source of the measurement, the event is narration, and medir:rag never reads the event log.

Both types are written OUT IN FULL at the emission points, not by constant: the inventory of docs/reference/events.md is generated by grepping type: '<x>.<y>' over the api's code, and a type behind a constant is invisible to it — the doc would pass green over an incomplete list. It is the same form execution.activated already uses. 'rag' enters PREFIXOS_DE_EVENTO (scripts/docs/generate.mjs) in the same commit.

4. Recording telemetry never brings down the search, and never fails silently.

The INSERT runs in a try/catch inside HybridSearchUseCase: whoever asks should not lose the answer because the measuring instrument fell. But the failure becomes a log with the origin classified (infra, ADR 0020) and searchId: null in the response — which is not the same as "found nothing": it is "there is nothing to attach a vote to", and the UI needs the two separated so as not to offer a control the api would refuse.

5. The vote requires the reference the search returned, and it is :direct.

An unknown searchId, or a chunkId that was not among that search's hits, is 400. It is not decorative rigor: the rank of the voted chunk is what separates "the index is poor" from "the WEIGHTS are wrong" — a poor index does not return the right chunk at any position, a wrong weight returns the right chunk at rank 7 —, and a vote without a rank produces a meaningless number.

The engine tool (rag_feedback) is category: :direct and does not become a proposed_action: rating a chunk has no external effect, and turning it into an action to approve would fill the queue with noise until nobody read the real ones anymore (the same rule that keeps rag_search/read_file out of the pipeline). The refusal comes back as an error tool-result, with the api's message, for the model to correct (RN-061/RN-163), never as a crash.

6. medir:rag fails for one thing only.

vector_available: false over the whole window: then what was measured is not the hybrid search, it is its lexical half, and calibrating a vector weight against that would be calibrating against a system that is not the one running. Everything else — precision@k, degradation rate, searches without a hit above the threshold, p50/p95 latency, rank distribution of what was voted useful — is report: it comes out in the table, it does not fail. Same mold as medir-execucao.ts.

Consequences

Calibration becomes possible, and is still not done. This stage does not change chunking, weights or threshold, on purpose: touching the numbers before accumulating measurement would destroy the baseline it exists to create. The item remains declared open in CLAUDE.md, now with the instrument next to it.

degraded and vector_available are two columns, and today they are redundant. degraded = !vector_available at this moment. The redundancy is declared in the schema instead of hidden, and the reason not to derive one from the other on read is that they do not answer the same question: vector_available is a fact about the embedding PROVIDER at the instant of the search — it is what makes medir:rag fail —, and degraded is the word of the CONTRACT with the engine (POST /internal/rag/search), whose definition can grow to other degradations without the provider having fallen. Deriving would tie the two forever.

The vote does not survive "Reindex now". rag_feedback.chunk_id references chunks with CASCADE. Reindexing deletes and recreates the chunks, so the judgment history goes with them. Keeping an orphan vote pointing to a chunk that no longer exists would measure nothing — but the price is real and is written in the schema: whoever reindexes loses the comparison base, and the measurement restarts.

precision@k measures what was JUDGED, not what was returned. The denominator is the number of voted hits with rank ≤ k, never k. A hit without a vote is unknown, not irrelevant, and counting it as irrelevant would make precision plummet on its own every time someone voted on fewer chunks — measuring the willingness to vote, not the quality of the search. The accepted consequence is that the metric only exists where there are votes: with none, it is null and the table says "not measured", never 0.

The internal contract with the engine grew in both directions. POST /internal/rag/search now accepts OPTIONAL sessionId/agent in the body (the api has no way to deduce either; a session it did not receive is a session that did not exist) and returns searchId plus a chunkId per hit. The chunkId is the only exception to the deliberate simplification of that DTO — without the searchId/chunkId pair the agent would have nothing to point to. When searchId comes null, the tool omits the ids and the invitation to vote, instead of offering the model a reference that would be refused.

One more tool in the prompt of six agents. rag_feedback enters the six registries that already had rag_search (dev/tools.ex, gates/qa_tools.ex, qa_estrategia_agent.ex, qa_performance_seguranca_agent.ex, appsec_agent.ex, harness/tools.ex). Real cost: one more spec block in the context of each turn. The alternative — only human feedback — would leave out the path most searches go through.

A viewer route that WRITES. POST /projects/:projectId/rag/feedback has the same role as search, and not maintainer: voting neither spends nor configures anything, it is observation, and restricting it would empty the only source of truth of the measurement. It is classified in docs/security-surface.md with this reasoning; the containment is the use case's reference validation, not the role.

No new agent, and no gate. The golden set and the rag-acertivo gate are the next stage, and depend on this one having accumulated data before they exist.