OpenAI Agents API vs a Custom Harness: When the Hosted Beta Is Enough

OpenAI put the Agents API into public beta on 10 September 2026. The pitch is a hosted Codex harness: durable cloud-agent sessions, orchestration, context compaction, recovery when a run dies mid-tool-call. You can run those sessions in OpenAI-hosted sandboxes or in self-hosted and partner environments.

That is a real product. It is also the third "you do not need your own agent runtime" announcement this year, after Responses-with-tools, the Assistants hangover, and every framework that promised to own the loop. Most teams asking us about it this week are really asking one question: do we stop maintaining the harness we already have.

Short answer: only if the harness is the bottleneck, and only if you are willing to live inside OpenAI's session model.

What the beta actually is

Not a model. Not an agent. A session runtime.

You describe work, tools, and (optionally) a sandbox. They keep the conversation, compact context when it gets long, retry or resume when a worker dies, and stream events back. Think "managed durable execution for an LLM loop," closer to a workflow engine than to LangGraph-in-your-repo.

That is useful. Compaction and recovery are the two pieces most homegrown loops get wrong, and they are the two pieces that show up as "the agent forgot the ticket" at 4am.

It is not useful as a place to hide your domain. Tools, permissions, evals, and the policy layer still belong to you. If those live only inside their session object, you have rebuilt Assistants with better branding.

When the hosted beta is enough

You do not have a harness yet, and the first product is OpenAI-only. Greenfield internal copilot, one model family, no compliance story that forbids vendor-side session state. Paying them to own compaction is cheaper than a staff engineer spending a quarter on it.

The work is long, tool-heavy, and already shaped like Codex. Multi-file refactors, repo-scoped tasks, cloud sandboxes. That is the workload the API was built around. Forcing it onto a six-second classification call is using a workflow engine as a function.

You want recovery more than you want to read the traces. If "resume from the last successful tool call" is the feature you would have built next, let them ship it.

Pair this with a real eval suite. Hosted does not mean evaluated. See agent eval harnesses.

When to keep (or build) a custom harness

You route across labs. Opus 5 on extraction, Sol on bounded code, Gemini on voice, a small model on classification. Their Agents API is an OpenAI session. A custom loop with a provider adapter is how the routing table stays a config file.

You need replay. Incident response, regulated workflows, "show me the exact tool args from Tuesday 14:03." If their event log is not your source of truth, or retention does not match your policy, you will rebuild tracing anyway. At that point the hosted runtime is a second system.

Tool I/O cannot leave your VPC. Retrieval over customer ledgers, production credentials, PHI. A self-hosted/partner environment option exists; read the data-flow diagram before you believe it. "Runs in your environment" still usually means their control plane sees metadata.

You already have a working loop. Replacing a harness that ships, has evals, and has on-call runbooks, because a beta launched, is how you lose a month and gain a migration. Wrap their API as one executor behind the same interface. Do not burn the old one.

Cost and lock-in, without the slide

You pay model tokens plus whatever the session/runtime line item is (confirm on the current price card; beta SKUs move). Compaction saves tokens. Recovery saves engineer hours. Both are real. Neither is free if every turn now includes their orchestration tax.

Lock-in is not the model. Lock-in is the session schema, the tool-calling dialect, and the sandbox image. Extract three things on day one:

1. Tool definitions you own (OpenAPI or MCP, in your repo).
2. Eval sets and traces you own (files, not a vendor UI).
3. A one-page adapter: run(session) -> events, so a Temporal/LangGraph/in-house loop can sit behind the same interface.

If you cannot point to those three, you are not "trying the beta." You are moving production into it.

A practical split we use

For MCP-shaped tools, keep the servers yours either way. See MCP vs native tool calling and OAuth-secured MCP.

Frequently Asked Questions

Is the Agents API a replacement for LangGraph or Temporal? For OpenAI-only LLM loops with compaction and resume, it can replace the LLM-loop part. It does not replace durable business workflows (payments, fulfillment, human queues). Keep those in a real workflow engine.

Can we run it in our VPC? They say self-hosted and partner environments are supported. Confirm where session state, tool payloads, and logs live, and who can read them, in writing, before a production token goes in.

What happens when we need Claude on one route? You add a second executor. If the product is coupled to their session object, that is a rewrite. If tools and evals are already yours, it is a config change.

Should we pause a custom harness project to wait for GA? No. Public beta is for learning the event model. Production SLAs, price, and data-flow guarantees are GA problems. Keep shipping on what you control.

---

FoundrySoft builds agent runtimes, eval harnesses, and MCP tool layers that stay portable across labs. See AI agent development or talk to an engineer.