The easiest way to approve an unsafe agent is to ask whether the model is “good enough.”
The model may be excellent. The system can still send duplicate emails, cross tenant boundaries, follow instructions from a retrieved document, retry a payment after an ambiguous timeout, retain secrets in traces, or continue working after the user cancels.
Production readiness is not a model property. It is evidence that the entire path from input to side effect behaves within an agreed risk tolerance.
Here is a 24-test review designed to fail loudly before a model touches a real tool.
Gate 1: Identity and Authority
1. Actor propagation
Can every tool call be traced to the authenticated human or service, tenant, agent identity, and run?
Pass evidence: a trace and receipt showing all identities without trusting model-supplied fields.
2. Least-privilege tool set
Does each agent see only the tools and scopes required for the current task?
Fail condition: a support-summary agent can discover payment, shell, file-write, or admin tools it never needs.
3. Resource-bound authorization
Is authorization evaluated against a canonical resource ID and action?
Pass evidence: cross-tenant, stale-role, and confused-deputy tests are denied by the executor, not merely by the prompt.
4. Expiring approval capability
Do high-risk actions require an approval bound to exact request, limits, approver, and expiry?
Fail condition: changing amount or target after approval still executes.
Gate 2: Tool and Side-Effect Safety
5. Schema plus semantic validation
Are values validated beyond JSON types—amount ranges, path containment, URL destinations, enum meaning, and payload size?
6. Stable idempotency
Can every write be safely retried with a runtime-generated logical action ID?
Pass evidence: duplicate delivery returns the same receipt; the same key with changed payload is rejected.
7. Ambiguous-outcome reconciliation
What happens when the tool commits but its response is lost?
Fail condition: every timeout is labeled failure and retried automatically.
8. Compensation contract
For multi-step effects, are compensating actions predefined, authorized, idempotent, and tested?
Compensation is not “ask the agent how to undo it.”
Gate 3: Bounded Execution and Recovery
9. Independent budgets
Are model calls, tokens, tool calls, branches, retries, cost, and wall-clock time bounded separately?
10. Deadline propagation
Does every child operation inherit the remaining absolute deadline?
Fail condition: each layer starts a fresh timeout and turns a 30-second request into minutes.
11. Cancellation propagation
Can user or operator cancellation stop new work, signal active tools, and safely handle late results?
12. Crash-and-resume safety
Has the workflow been crashed before and after every effect boundary, then resumed twice?
Pass evidence: state converges and no required at-most-once effect duplicates.
Gate 4: Injection and Isolation
13. Untrusted-content labeling
Are user text, web pages, retrieved documents, email, and tool outputs treated as data rather than authority?
14. Primitive reachability map
For each untrusted source, which tool primitives can it influence?
Microsoft’s 2026 Semantic Kernel research showed why this matters: model-controlled input reached unsafe evaluation and host file-write paths, turning prompt injection into execution.
15. Runtime isolation
Do code and browser tools run with isolated identities, restricted filesystem, bounded CPU/memory/time, and default-deny egress?
16. Secret and output containment
Can the agent read secrets, signed URLs, raw credentials, or sensitive tool fields it does not need? Are tool outputs filtered before returning to the model?
Gate 5: Data, Memory, and Evaluation
17. Data minimization
Are prompts, traces, replay stores, and logs allowlisted, redacted, encrypted, retained briefly, and access-controlled?
18. Memory lifecycle
Does every durable memory have provenance, scope, confidence, sensitivity, expiry, correction, and deletion semantics?
19. Trajectory evaluation
Do evaluations inspect tool selection, arguments, policy decisions, side effects, final state, latency, and cost—not only the final answer?
20. Repeated-trial reliability
Are high-impact tasks tested across repeated runs and realistic failure conditions?
Fail condition: a one-of-five success is reported as “80% reliable” because the best attempt was selected.
Gate 6: Observability and Operations
21. Structured trace reconstruction
Can an operator reconstruct state transitions, model/tool versions, policy decisions, approvals, receipts, and terminal reason without hidden chain-of-thought?
22. Release and rollback gate
Are model, prompt, retrieval, tool-schema, and policy changes evaluated against regression thresholds before rollout, with a tested rollback?
23. Kill switch and containment
Can operators revoke agent credentials, disable a tool, stop new runs, quarantine current runs, and block egress quickly?
24. Incident evidence
Are logs sufficient to define the vulnerable window, identify affected runs and resources, find suspicious host/network behavior, rotate reachable credentials, and notify owners?
If the answer appears only in a dashboard screenshot and cannot be reproduced, the test has not passed.
Use Risk Tiers, Not One Universal Bar
A document summarizer and a payment agent should not have identical controls.
Classify tasks by authority:
Tier 0: generate text only
Tier 1: read approved data
Tier 2: create reversible drafts or reservations
Tier 3: perform external side effects
Tier 4: high-impact, regulated, or irreversible effects
The higher the tier, the stronger the evidence:
- narrower tool scopes;
- stricter approval;
- stronger isolation;
- more repeated trials;
- zero-tolerance safety metrics;
- shorter credential lifetime;
- deeper incident readiness;
- smaller initial rollout.
Risk tier must follow the most powerful reachable primitive, not the product’s marketing description. A “research assistant” with shell, browser, file-write, and cloud credentials is not read-only.
Make Every Test Produce an Artifact
A review meeting creates confidence. A review artifact creates accountability.
For each test, record:
{
"test_id": "PRR-12",
"risk_tier": 3,
"environment": "staging-isolated-v7",
"agent_version": "support-31",
"tool_schema_version": "tools-19",
"procedure_ref": "test://crash-resume/effect-gap",
"evidence_refs": ["trace://...", "receipt://..."],
"result": "PASS",
"owner": "agent-platform",
"reviewed_at": "2026-07-30T09:00:00Z",
"expires_at": "2026-08-30T09:00:00Z"
}
Evidence expires. A prompt change may affect tool choice. A new tool changes reachability. A framework upgrade can change resume semantics. A permission update can invalidate least privilege.
Re-run the affected tests on change. Do not stamp “production-ready” permanently on a moving system.
Connect the Review to Risk Management
NIST’s AI Risk Management Framework uses Govern, Map, Measure, and Manage. A useful production review touches all four:
- Govern: owners, risk tolerance, approvals, and accountability;
- Map: users, data, tools, effects, dependencies, and harm paths;
- Measure: tests, evals, red-team results, latency, cost, and uncertainty;
- Manage: release gates, monitoring, containment, rollback, and incidents.
OWASP’s agentic-security work provides a complementary threat lens across excessive agency, tool misuse, identity and privilege, supply chain, memory, and related risks.
Neither framework means “complete this checklist and the agent is safe.” They help teams ask systematic questions. Your 24 tests must be adapted to the actual tools, users, data, and consequences.
A Readiness Decision Has Four Outcomes
Avoid the binary theater of approved/not approved.
Use:
- PASS: evidence meets the tier’s bar;
- CONDITIONAL: limited rollout with explicit constraints and owner;
- INCONCLUSIVE: insufficient or stale evidence; do not expand;
- FAIL: a known control or invariant is missing.
“Inconclusive” is not a polite pass.
For a new agent, start with no side effects, then read-only tools, then reversible drafts, then a small canary for approved effects. Authority should be earned through evidence.
The best model demo asks, “What can the agent do?”
The production readiness review asks the harder question:
What can the system prevent, prove, recover, and contain when the agent is wrong?
Do not let a model touch a real tool until the answer exists in artifacts, not optimism.