Non-blocking human-in-the-loop agents: re-engineering agentic runloops and state machines with MRTR

How MCP 2026-07-28 Multi Round-Trip Requests let enterprise agents pause for human approval without long-lived server connections — with a LangGraph and agentgateway design walkthrough.

If you have built an AI agent that can call tools — query a database, open a ticket, run a migration — you have already met the hard part of enterprise adoption: when the agent must stop and ask a human before it acts. Approving a destructive change is not optional in regulated environments. The awkward question is how you pause mid-tool without freezing your infrastructure around that pause.

The Model Context Protocol (MCP) is the open standard many agent stacks now use to talk to tools. Think of it as a shared contract between an agent host and the services that expose capabilities (search, tickets, databases, and so on). Until recently, many production HITL stacks leaned on desktop-era implementation patterns: open a session, keep a long-lived connection, and push follow-up questions over that pipe while the server held work in memory. Streamable HTTP made server to client streams available, but parking a paused execution frame was the common implementation choice on top of that.

The 2026-07-28 MCP specification breaks that model by providing a cleaner first-class alternative. The protocol core becomes stateless: each request stands alone. For human-in-the-loop flows, the important new piece is Multi Round-Trip Requests (MRTR). Instead of holding a network connection open while an operator thinks, the server finishes the HTTP response with "I need input," hands the client an opaque continuation value, which the demo supporting this article further protects with an HMAC signature, and walks away. When the human answers, the client retries. Any healthy server replica can pick up where the first one left off.

This article explains why the old pause failed at scale, how MRTR redesigns both the tool-side and agent-side state machines, and what a working DevOps demo actually proves — without turning this into a clone of the repository README. The reference code lives at mcp-mrtr-devops-demo.

Why "just ask the human" used to break the network

Picture an autonomous DevOps agent asked to run an emergency migration on production cluster prod-db-01. The script is V004__drop_legacy_users.sql. Before any DROP executes, a human must confirm.

In many earlier HITL implementations on MCP Streamable HTTP, that confirmation was implemented as a transport problem. The client opened a protocol session, kept a long-lived server-to-client stream (often Server-Sent Events, or SSE — a one-way HTTP channel the server can push messages on), and invoked the tool. Mid-call, the server pushed an elicitation — a structured "please fill this form" request — over the open stream. A common implementation choice was to leave the tool's execution frame paused in that process's memory so when the operator answered, the reply had to reach the same server instance for that frame to resume.

That design is fine on a laptop. It is fragile behind an enterprise load balancer. The gateway must "stick" the client to one pod (session affinity). Idle proxies time out quiet streams. Rolling deploys and autoscaling kill the pod that was holding the pause. Meanwhile the open connection sits in memory for as long as a human takes to read the dialog — seconds or minutes that your connection pools cannot usefully spend waiting.

In enterprise system architecture, human approval is a control-plane concern — a deliberate gate in the execution path — not something the network layer should absorb by holding sockets open. Coupling that gate to a long-lived connection turns a governance checkpoint into a load-balancer and capacity failure mode.

What the new specification changes

MCP 2026-07-28 retires protocol-level sessions. There is no mandatory initialize handshake and no Mcp-Session-Id that pins you to one backend. Protocol version and client capabilities travel with each request in a `_meta` object — small structured metadata attached to the JSON-RPC body — and clients should also identify themselves with `clientInfo`. Authentication and caller identity still come from the authorization layer. If you want to learn a server's capabilities up front, there is an optional server/discover call; you do not need it just to invoke a tool.

On the wire, Streamable HTTP also requires a few HTTP headers that name the method and tool (Mcp-Method, Mcp-Name, plus the protocol version). That sounds pedantic until you run traffic through a gateway such as agentgateway: once those headers are checked against the request body, the perimeter can route and apply coarse policy without digging through every JSON body. Fine-grained authorization still needs the authenticated caller, the operation, and the resource. That fits the perimeter layer we described in Governed Run Loops.

MRTR is how interactivity works once sessions are gone. When a tool needs mid-call input, the server does not push over a held stream. It completes the current HTTP response with an interim result. Three fields matter:

• resultType — tells the client whether this is a finished answer (complete) or a yield (input_required).
• inputRequests — describes what the client should collect, typically a form elicitation with a JSON Schema the UI can render.
• requestState — an opaque continuation handle minted by the server: a sealed snapshot of "where we were," not a session cookie.

The client gathers answers in its own UI or terminal, then re-issues the same tool call with inputResponses and the echoed requestState. A new JSON-RPC request id is used (normal JSON-RPC practice); continuity lives in those params, not in transport affinity. When the work is done, the server returns resultType: "complete".

The protocol also constrains when a server may ask. A server may only solicit input while it is actively handling a client-initiated request. That chain of custody keeps agents from spontaneously popping dialogs at the user.

The mental model is the whole story:

Common approach before: pause the thread, keep the socket.
With MRTR: serialize the state, close the socket, resume on any instance.

Two state machines, deliberately separated

Those earlier stacks often fused "agent waiting for a human" with "server holding a network connection." MRTR makes the split explicit: continuation in `requestState`, human think-time off the wire.

On the server: continue, do not suspend

A destructive tool no longer blocks on network I/O. It inspects the script, decides confirmation is required, mints a continuation, returns input_required, and ends the response. Later, on a fresh request, another process — possibly on another machine — verifies the continuation, checks the human's answers, and either applies or denies.

How you store that continuation is an engineering choice. You can pack the step and bindings into a signed blob and send it entirely in requestState (no shared database; excellent for horizontal scale). Or you can park rich context in Redis or similar and put only a handle on the wire (smaller payloads; you now operate a cache with TTLs).

Either way, on the retry path you must treat requestState as attacker-controlled input. Sign it so clients cannot flip the cluster name or step flags. Bind the tool arguments (and ideally the authenticated user) into what you sign, and reject mismatches. Expire the token so approvals cannot be replayed forever. Fail closed: an invalid or stale continuation denies the action — it never "best efforts" into production.

On the agent: a non-blocking harness

The agent runloop — the loop that plans, calls tools, and feeds results back to the model — should branch on resultType, not on whether a streaming socket is still open. With LangGraph, a natural shape is: call the model, call the tool, and if the tool yields, interrupt for human input, then retry the tool with the answers and the continuation token before finishing.

The harness is responsible for turning an elicitation schema into a UI (or terminal prompts), packaging answers under the keys the server expects, echoing requestState exactly, and capping how many round trips a single tool call may take so a buggy server cannot loop the operator forever. Agent-side checkpointing (so your graph can resume after a page refresh) is useful — but it is not a substitute for server-side verification of requestState. One is UX continuity; the other is trust.

That separation is the difference between a demo that works once and a fleet that survives a pod recycle while someone is still reading the confirmation dialog.

What the demo is designed to show

The open-source demo caldeirav/mcp-mrtr-devops-demo is a deliberately small production shape, not a full platform. A LangGraph agent asks an MCP tool apply_db_migration to run a migration. Tool traffic always goes through agentgateway in statefulMode: stateless — meaning the proxy is not allowed to invent sticky MCP sessions. The MCP server is a FastAPI endpoint speaking the 2026-07-28 request shape. Continuations are HMAC-SHA256 signed with a five-minute lifetime. The gateway also exports OpenTelemetry traces (OTLP) so each short HTTP round trip can show up as inspectable spans in Jaeger — optional for the core HITL story, useful when you want the pause/resume pattern visible outside the terminal.

The story the demo walks is the enterprise one: the agent proposes a destructive script; the server yields; the operator confirms; the agent retries; the migration completes (simulated) or is denied — and at no point does anyone hold an SSE stream open waiting for the human.

Server pattern: yield, then verify on resume

On the first call, if the script looks destructive and there is no continuation yet, the server builds an elicitation and mints requestState, then returns immediately:

def _build_input_required(cluster_id: str, script_name: str) -> dict[str, Any]:
    token = mint_request_state(
        secret=_hmac_secret(),
        cluster_id=cluster_id,
        script_name=script_name,
    )
    elicit = ElicitRequest(
        params=ElicitFormParams(
            message=(
                f"Confirm destructive migration on {cluster_id} ({script_name}). "
                f"environment_tag must be one of: {', '.join(ENVIRONMENT_TAGS)}"
            ),
            requestedSchema=build_confirm_drop_schema(),
        )
    )
    return InputRequiredResult(
        inputRequests={ELICITATION_KEY: elicit},
        requestState=token,
    ).model_dump()

The important design choice is what goes into the token. The demo signs cluster_id, script_name, and an issuance timestamp. On resume it verifies the HMAC, checks the TTL, and rejects tokens that do not match the arguments on the new request. Tampering, expiry, or a forged signature all fail closed before any "apply" path runs.

The response the client sees looks like this — a finished HTTP 200 JSON-RPC result, not a parked connection:

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "resultType": "input_required",
    "inputRequests": {
      "confirm_drop_form": {
        "method": "elicitation/create",
        "params": {
          "mode": "form",
          "message": "Confirm destructive migration on prod-db-01 (V004__drop_legacy_users.sql). environment_tag must be one of: dev, staging, prod",
          "requestedSchema": {
            "type": "object",
            "properties": {
              "confirm_drop": {
                "type": "boolean",
                "description": "Confirm DROP/destructive ops"
              },
              "environment_tag": {
                "type": "string",
                "enum": ["dev", "staging", "prod"],
                "description": "Target environment tag"
              }
            },
            "required": ["confirm_drop", "environment_tag"]
          }
        }
      }
    },
    "requestState": "<payload_b64>.<hmac_hex>"
  }
}

The same yield through agentgateway looks like this in the MCP Tool Playground — apply_db_migration on prod-db-01 with V004__drop_legacy_users.sql, returning HTTP 200 with resultType: "input_required", the elicitation schema, and a requestState handle:

Client pattern: branch on resultType, interrupt, retry

The LangGraph graph is intentionally boring — and that is the point. After call_tool, routing looks only at the protocol discriminator:

def route_after_tool(state: AgentState) -> Literal["human_input", "done"]:
    if state.get("error"):
        return "done"
    result = state.get("last_result") or {}
    if result.get("resultType") == "input_required":
        return "human_input"
    return "done"

The human_input node uses LangGraph's interrupt() so the graph parks without blocking the MCP server. When the operator answers, retry_tool issues a new HTTP POST through the gateway with the same tool arguments, plus inputResponses and the echoed requestState. Nothing on the server needs the original thread — only the shared signing secret and the arguments on the wire.

The demo runs a single MCP server, so what you see live is those two independent requests through a stateless gateway rather than a retry that lands on a different replica. Cross-replica resume is a property of the design: `requestState` is a self-contained HMAC-signed snapshot of the pause (not a pointer into one process's memory), the gateway stays in `statefulMode: stateless` and never invents an `Mcp-Session-Id`, and any instance that shares the signing secret can verify the token, re-bind the tool arguments, and continue. Put several replicas behind the same round-robin edge and the same retry path works without a sticky conversation glued to one pod.

Because each retry is a fresh HTTP POST, distributed tracing fits the architecture naturally. With agentgateway's OTLP export pointed at a local Jaeger all-in-one, tool round trips appear as ordinary spans — here a POST into the gateway nesting a tools/call on the devops-migration target — instead of one long-lived stream that spans the entire human think-time:

For setup and run instructions, use the repository README. The harness's banded console output (TRACE, AGENT, HITL, SEP) remains useful for watching protocol fields on a single happy path; Jaeger is the complementary view when you want the same round trips as spans.

Design takeaways for production

A few rules of thumb follow directly from this pattern:

• Keep HITL state out of the transport. The client or harness must preserve and echo requestState on retry so that a replica with access to the required verification material or shared state can resume. The model and human operator do not need direct access to the value.
• Keep the gateway honest. If your proxy still pins MCP sessions, you have reintroduced sticky routing under a new name. The demo's project rules forbid Mcp-Session-Id for that reason.
• Treat the continuation like a capability. Protect integrity (HMAC or AEAD), bound time, bind arguments (and ideally the authenticated principal), and fail closed on every resume.
• Separate agent checkpoints from MCP continuation. Graph memory helps the UX; server verification of requestState is what makes the action safe.
• Cap client round trips. A misbehaving tool must not turn "ask once" into an infinite confirmation loop.
• Route on headers at the edge. Prefer gateways that steer on Mcp-Method and Mcp-Name for routing and coarse policy, and reserve deep body inspection for audit enrichment rather than basic steering.

MRTR does not replace long-running async work (that is moving into the Tasks extension), and it does not replace your governance stack. It replaces the fragile pause — the part that used to couple human think-time to socket lifetime.

Demo repository: github.com/caldeirav/mcp-mrtr-devops-demo. Spec announcement: The 2026-07-28 Specification.