A demo can look good and still be impossible to trust.
Five people try it. The answers look reasonable. Everyone moves on.
Then the prompt changes. The model changes. Documents get added. Retrieval changes. Six months later someone asks a simple question:
Is it still working as well as it did when we approved it?
A lot of teams cannot answer that repeatably.
That is what evals are for.
I separate the problem into three parts.
Before release, I want repeatable tests against known cases. For RAG that may mean: did the expected document appear, did a forbidden document appear, did the answer include a required fact, did latency stay inside the limit, and did cost move unexpectedly?
After release, I want observability. Latency, errors, traces, token usage, cost, and sampled quality signals tell me what the live system actually did.
Then I want the useful traces to feed back into the test set. A production failure, strange trajectory, or unexpectedly good case should become something I can reproduce and evaluate the next time the prompt, model, retrieval, policy, or harness changes.
That creates a loop:
run / production trace
-> inspect failure or useful case
-> add or refine eval
-> change model / prompt / retrieval / harness / policy
-> rerun benchmark / release gate
-> promote or block
-> observe again
Observability, evaluation, and release assurance belong together, but they are not the same thing.
A dashboard tells me what happened.
An eval tells me whether the behavior met the expectation.
A release gate decides whether a known regression should be allowed to ship.
For LLM applications, there are several useful evaluation approaches. Human review is still important when judgment matters. LLM-as-judge can help with semantic quality at scale, but it is still another probabilistic model. Golden datasets give you a stable set of cases to rerun. Regression tests tell you when a change broke behavior you already cared about.
For RAG, I still want the basic retrieval and answer signals: precision, recall, required facts, citations, grounding, latency, and cost.
But I ran into another question that a single quality score does not answer:
Was this evidence correct for this user?
That distinction matters in enterprise systems.
An employee asks about executive compensation. A restricted compensation document may be the most relevant document in the corpus. A perfect retrieval score does not make it authorized.
The application should remove that context before generation. The evaluation should also be able to state that the source is forbidden for this principal and fail if it appears.
That led me from a generic RAG release-gate experiment to a more specific project I am calling RAGAssure.
The idea is still small. A case contains normal quality expectations — expected evidence, required facts, retrieval behavior, latency, cost — but it can also carry a principal such as a tenant, role, or department and declare evidence that must never appear for that principal.
Then the result is not just a quality score.
It can say:
principal: acme / employee
required evidence: present
forbidden evidence: absent
latency budget: passed
release verdict: PASS
Or, when something leaks:
principal: acme / employee
forbidden evidence: executive-compensation-2026
release verdict: BLOCK
I am not trying to replace standard RAG evaluation. I want quality, access, and release evidence to be testable together.
The deterministic checks should own hard rules where possible. LLM-based evaluators still have a place for faithfulness, relevance, completeness, and other semantic judgments. I just do not want another model to be the only thing deciding whether a known access or policy rule passed.
Agentic systems make the trace even more important.
I find the model + context + harness separation useful here. The harness runs the loop: what context reaches the model, when a tool runs, how state changes, whether the system retries, and when it stops. A final answer can look fine while an intermediate tool call, argument, retry loop, or policy decision was wrong.
That is why future agent evaluation has to look at the trajectory, not only the final text.
I have worked with the usual observability pattern — traces, metrics, and logs — for platform services. AI adds quality and behavioral evidence to that picture. A system can be healthy from an infrastructure point of view and still be wrong.
This is the end of the five-part Enterprise AI series.
The thread across all five posts is simple: the hard part starts when the demo has to become a system — one with constraints, ownership, production controls, and evidence that it still works.
That last question became RAGAssure, which is now public as an open-source 0.1.0 alpha and installable from PyPI. I am using the Secure Enterprise AI Assistant as its first public system under test: the application owns the runtime authorization boundary, and RAGAssure verifies that the expected evidence behavior still holds after a change.