The demo ends when the agent clicks the button. The engineering starts when the request times out and nobody knows whether the button worked.
Consider an agent preparing a supplier comparison. It reads documents, runs a calculation, generates a report, and offers to email the result. Nothing about that workflow sounds especially dangerous. Then a worker restarts after the email provider accepts the message but before the application records success.
Should the replacement worker send it again?
A better prompt cannot answer that question. The system needs a definition of the action, a durable record of its intent, and a way to reconcile an uncertain outcome. Otherwise, a perfectly reasonable retry becomes a duplicate external write.
That is what I mean by deterministic infrastructure for AI agents: not forcing a model to produce identical sentences, but making authority, execution state, and recovery explicit enough that a probabilistic component cannot quietly redefine them.
The unit of design is an action, not a conversation
A conversation is a useful interface. It is a poor transaction boundary.
“Research these suppliers and send me a recommendation” can contain several operations with different permissions. Reading a public document, executing generated Python, accessing an authenticated account, and sending an email should not inherit one undifferentiated authorization just because they belong to the same chat turn.
I start by separating what the model proposes from what the application permits. The model can suggest an action and supply arguments. The execution layer resolves identity, checks the relevant resource, validates the arguments, applies the current policy, and decides whether to execute, deny, or request approval.
This separation makes a useful distinction possible: the agent may understand that an email tool exists without having permission to send this particular email.
My objection to “just add guardrails” is that it often leaves the most important noun unspecified. Guard which boundary, against which failure, with whose authority?
A content filter, a tool schema, an approval screen, and a network policy are different controls. None substitutes for the others.
Deterministic does not mean infallible
The word can become misleading when it is used as a synonym for reliable.
A deterministic policy evaluator can return the same decision for the same recorded inputs. That does not make the policy correct. A state machine can reject an invalid transition. That does not make an external service available. A replay can reconstruct an execution trace without reproducing the model’s exact output.
The achievable goal is narrower and more valuable: predictable rules around unpredictable proposals.
For each privileged action, I want an answer to five questions: who requested it, what exact operation was authorized, which state that authorization depended on, how long the authority lasts, and how the outcome will be established.
Those answers belong in records the application understands, not only in prose the model has generated.
A small execution contract goes a long way
A useful starting point is an action envelope. This example is deliberately small; it illustrates the information boundary, not a complete security implementation.
{
"action_id": "act_7f42",
"run_id": "run_b619",
"principal_id": "user_23",
"operation": "report.publish",
"resource_id": "report_81",
"resource_version": 4,
"arguments_digest": "sha256:example-digest",
"policy_version": "publish-policy-v3",
"deadline": "2026-10-06T12:30:00Z",
"approval_required": true
}The important fields are not the particular names. They are the relationships.
The resource version makes the authorization specific to an object state. The arguments digest makes it specific to the proposed operation. The deadline prevents a forgotten task from retaining indefinite authority. The policy version makes the decision explainable after a configuration change.
A digest alone proves nothing about who approved an action. The approval record still needs authenticated identity, server-side validation, and protection against reuse. Similarly, a deadline written into JSON is only useful when the actual executor checks it.
Keep a durable action identifier separate from an execution-attempt identifier. Retries of the same intended action should preserve the former while receiving new attempts. A genuinely new user instruction should not accidentally reuse an old deduplication key.
Idempotent APIs are valuable precisely because a timeout does not tell the caller whether a side effect occurred. AWS’s engineering discussion of idempotent operations addresses this ambiguity at the API boundary.[1]
Separate trust before separating services
I do not begin with a target number of microservices. I begin with the capabilities that should never share ambient access.
An authenticated browser contains session state that may grant access to an account. A generated-code runtime executes instructions that should be treated as untrusted. The control service can authorize work. An artifact store can expose outputs to people who were not present during execution.
Putting these in different processes is a start. Giving those processes the same credentials, writable volumes, and unrestricted network access can undo much of the benefit.
The control service should issue narrow work requests. The browser should receive access to the required identity and destinations, not the application’s entire secret environment. Generated code should receive an approved input file or a scoped handle, not the database password used to obtain it.
Artifacts should move through a promotion step: identify the producing run, validate the type and size, record an immutable version, and authorize access before sharing.
This is also where deployment convenience can become a trap. Docker provides useful isolation mechanisms, but daemon access and host configuration are security-sensitive. A mounted Docker socket is not an ordinary application dependency.[2] Where the threat model includes hostile generated code, a sandboxed runtime can provide an additional boundary; it still has its own compatibility and security limits.[3]
The architecture can remain easy to deploy without making every component equally powerful.
A state machine is more useful than a confident status message
For a side-effecting operation, model at least these distinctions:
proposed -> awaiting_approval -> authorized -> executing
-> succeeded
-> failed
-> outcome_unknownThere are additional transitions for expiry, rejection, and cancellation. The crucial state is outcome_unknown.
Without it, the application tends to force ambiguous failures into “failed,” which invites an unsafe retry, or “succeeded,” which invents an outcome. A timeout after submission is different from a validation error before submission.
A restart should consult durable action state and reconcile with the external service where possible. It should not ask a fresh model invocation to infer whether the previous worker probably finished.
Cancellation deserves the same precision. Once cancellation is recorded, the dispatch gate should reject new work for that run. It may not undo an operation already accepted by another system. The interface should say which work was stopped, which work completed, and which outcome still needs reconciliation.
A truthful partial result is better than a reassuring fiction.
Ownership must survive a worker restart
Suppose worker A owns a run. Its lease expires during a pause. Worker B takes over. Worker A then resumes.
A heartbeat tells us whether a worker has been heard from recently. It does not, by itself, prevent that old worker from writing.
For writes under the application’s control, a generation number or fencing token can make stale ownership rejectable. The durable store accepts a state transition only when the writer presents the current generation and the expected previous state. Claiming ownership, allocating that generation, and recording the transition need transactional or otherwise atomic semantics.
For external writes, the problem is harder. A third-party service may not understand the generation. An action gateway can centralize dispatch and ownership checks. It still needs destination-supported idempotency, outcome reconciliation, or a deliberate refusal to retry automatically when the effect cannot be established. A distributed lock does not magically reach across an API boundary.
This is the sort of detail that makes an architecture credible. “We use leases” is the beginning of an explanation, not the end.
Replay the evidence; do not replay the side effect
A useful audit trail includes the action envelope, policy outcome, tool version, attempt identity, input references, output references, and relevant state transitions. It should not indiscriminately collect credentials, private page contents, or hidden model reasoning.
The trace needs to answer what the system observed and did. It does not need to pretend to expose every internal computation of the model.
There are two different replay modes worth naming. An inspection replay rebuilds the visible sequence from recorded events and saved outputs. An execution replay runs code again in an isolated test environment with external writes disabled or replaced by fixtures.
Confusing the two is dangerous. A “debug this run” button should not send the email again.
Keep enough provenance to identify an input and its retrieval time. Make retention, redaction, and access control explicit. An immutable record is not an excuse to retain sensitive information forever.
What I would test before adding more autonomy
Start with a small failure suite rather than a larger collection of prompts.
One test sends the same logical action twice and checks that the external outcome is not duplicated. Another expires an approval, then verifies that execution fails even though the model still has the old approval text in context. Another cancels a run while a child worker is active and checks that the child cannot publish a late result as current.
For isolation, test that generated code cannot access a browser profile or control-service credentials. For artifacts, verify that a user cannot fetch another user’s output by guessing its identifier. For recovery, crash the worker between external acceptance and local success recording.
These tests do not prove the system is secure. They make the intended boundaries executable and expose regressions that a successful demo will never show.
The most useful result of this exercise may be discovering that an operation is not safely automatable yet. That is architectural information, not a failure of ambition.
Start with one trustworthy action
It is tempting to build the entire agent platform before shipping a narrow workflow. My preference is the opposite: make one consequential action understandable from proposal through recovery.
Give it a typed request, explicit authorization, bounded execution, a durable outcome, and an honest failure path. Then reuse those contracts across more tools.
The model will improve. Providers will change. A new orchestration library will make the demo shorter. None of that removes the need to know whether an action was authorized, whether it happened, and what the system should do next.
That is the infrastructure worth keeping.
Technical notes
[1] AWS Builders’ Library: making retries safe with idempotent APIs.