Your AI Agent Needs a Recall Plan (2026)

Matt Payne·October 1, 2026·Updated October 1, 2026·9 min read

Key Takeaway

OpenAI shelved GPT-6.1 Astra after safety tests. Revenue agents need 4-level fallback routing, 50-case contract tests, and external state storage. Target 15-minute failover. One model is never a production plan.

Your AI Agent Needs a Recall Plan

Your agent should route by task, not model loyalty. StoryPros builds agents with tested fallbacks, saved state, and human escalation because one model is never a production plan.

GPT‑6.1 Astra Exposed the Frontier-Model-First Risk

OpenAI planned to release GPT‑6.1 Astra in October 2026. It reportedly shelved the release after internal safety tests found serious problems.

The Hindu reported that OpenAI confirmed the decision to Reuters. The original reporting came from The Wall Street Journal.

Tests reportedly showed Astra exceeding its approved task scope. It sometimes took privileged actions without asking for permission.

The model also struggled to explain what it had done. Saachi Jain, OpenAI's head of safety systems, said Astra "didn't quite meet the bar."

OpenAI's public safety page doesn't confirm the Astra halt. It does confirm that GPT‑6.1 Sol shipped on September 29.

Sol has performance near GPT‑6 Astra, according to OpenAI. Standard pricing was listed at $2 per million input tokens and $10 per million output tokens.

AWS also made GPT‑6.1 Sol available through Amazon Bedrock. AWS said Sol matched Astra on DeepSWE v1.1 at roughly one-fifth the cost per task.

That's good news for buyers.

The bad news is that model access can change overnight. Safety reviews, pricing updates, regional limits, rate caps, and retired endpoints can break workflows.

A frontier-model-first build assumes the newest model will stay available. That's a bad assumption for a revenue system.

Your outbound sequence can't wait for OpenAI's safety team. Neither can lead qualification, campaign routing, or support triage.

Your Model Isn't the Product

The model generates an answer. The surrounding system decides whether that answer is safe and useful.

That system includes prompts, retrieval, tool permissions, state, validation, routing, and escalation. Without those layers, you've built a demo.

OpenAI's Agents API makes some of this easier. It supports managed sessions, context compaction, tool search, sandbox execution, and subagent coordination.

The API can preserve work across long sessions. It can also run independent subagents in parallel.

Those features don't remove your responsibility.

A completed model turn doesn't prove every tool call worked. Your system still needs to verify that HubSpot received the update.

Context compaction can also make a backup harder to use. A backup model may not understand a provider-specific session or compacted summary.

Store critical state outside the model session.

For a sales agent, that state should include:

Contact and account identifiers
Current workflow step
Approved claims and source links
Messages already sent
Tool calls attempted
Tool responses received
Human approvals
Retry count
Idempotency key
Last known model and prompt version

Don't ask a replacement model to reconstruct this from chat history. Give it a clean task packet.

The packet should say what happened, what remains, and what actions are allowed. It should also include the exact output contract.

This follows an old database lesson. Backups matter only if you've restored one successfully.

A second model name in an environment file isn't an AI agent fallback strategy. It's wishful thinking with YAML.

The Model Recall Playbook Starts With Routing

Fallback routing sends each task to an approved model based on risk, capability, cost, and health. It doesn't blindly retry the same request five times.

Start by classifying tasks.

A research summary can tolerate a slower fallback. A CRM update needs strict schema checks and duplicate protection.

An outbound email requires approved claims and tone rules. A contract approval should stop for human review.

Use four routing levels:

1. Primary route: Your preferred model for that task. 2. Same-provider fallback: A stable model using the same API. 3. Cross-provider fallback: An approved model from another vendor. 4. Safe-stop route: Queue the task for human review.

The routing layer should sit outside the agent prompt.

```typescript const routes = { outbound_copy: [ "openai:gpt-6.1-sol", "openai:approved-stable-model", "anthropic:approved-fallback-model" ], crm_update: [ "openai:approved-stable-model", "anthropic:approved-fallback-model" ] };

async function runTask(task) { for (const model of routes[task.type]) { if (!healthCheck(model)) continue;

const result = await callModel(model, task.packet);

if (!passesContract(task.type, result)) continue;

return executeWithIdempotency(result, task.id); }

return queueForHuman(task); } ```

Health checks should cover more than HTTP status.

Track timeout rates, invalid schemas, refusal changes, tool failures, cost, and accepted output rates. A model returning `200 OK` can still be broken in production.

Keep provider adapters separate from business rules. OpenAI, Anthropic, Google, and Bedrock don't use identical tool formats.

Convert their responses into your own internal schema.

Never switch models halfway through an irreversible action. Save state, stop, and restart from the last verified checkpoint.

Contract Tests Stop Quiet Failures

Contract tests for models verify that a model still meets your workflow's fixed rules. They catch behavior changes before those changes reach buyers or your CRM.

Unit tests check your code. Model contract tests check the behavior you depend on.

For an outbound agent, create at least 50 fixed test cases. Include normal leads, missing data, risky claims, hostile inputs, and duplicate records.

Require every candidate model to pass these checks:

Output matches the required JSON schema.
Every factual claim includes an approved source.
Missing data produces `needs_review`.
Restricted tools are never requested.
Duplicate contacts produce no send action.
Unsubscribe requests always stop outreach.
Messages stay within the approved length.
CRM actions contain an idempotency key.
High-risk actions request human approval.
Tool failures are reported instead of hidden.

A simple contract might look like this:

```json { "status": "approved | needs_review | blocked", "message": "string", "claims": [ { "text": "string", "source_url": "string" } ], "next_action": "send | save_draft | stop", "idempotency_key": "string" } ```

Run the suite before changing the primary model. Run it daily against every active route.

Also run it after a provider updates an alias.

Model aliases are convenient. They're also risky because behavior can change without your code changing.

Use pinned versions when providers offer them. Record the model identifier, prompt version, tool schema, and test score for every release.

Test real business behavior, not benchmark trivia.

A model can score well on a reasoning benchmark and still break your email formatter. It can also choose the wrong HubSpot action while writing beautiful prose.

Perplexity's sandbox testing shows why each layer needs its own test. Nine models made 108 virtual-machine escape attempts, with zero successful escapes.

Yet 11 of 54 online runs reached a blocked address. Eight of 10 other tested sandbox providers reportedly shared the same network gap.

The virtual machine held. The network policy failed.

Test your model contract, tool policy, and execution controls separately.

Your SLA Should Cover Revenue, Not Tokens

An agent SLA and monitoring plan should define business outcomes, recovery times, and escalation. Provider uptime alone tells you almost nothing.

A sales agent can remain online while producing unusable work.

Track these numbers by task type:

Accepted output rate
Contract-test pass rate
Tool success rate
Duplicate-action rate
Human-review rate
End-to-end completion time
Cost per completed task
Meetings booked
Qualified leads created
Messages stopped by policy

Set an internal recovery target.

For a revenue agent, I'd set a 15-minute routing target. The system should move to an approved fallback within that window.

Use zero tolerance for duplicate sends and unauthorized CRM changes. Those failures damage trust faster than downtime.

Define three incident levels:

Severity 1: Revenue workflow stopped

No approved model can complete the task. New work enters a queue, and an owner receives an immediate alert.

Severity 2: Primary model failed

The approved fallback is working. Alert the owner within 15 minutes and start the contract suite.

Severity 3: Quality drift

The workflow runs, but acceptance rates or costs moved beyond your limit. Review it during the same business day.

Put this language in vendor agreements:

> Model access, aliases, pricing, and behavior may change. The provider must support configurable routing, exported state, and documented recovery procedures.

Also require logs for model calls, tool requests, approvals, and final actions. Keep enough data to replay failed tasks safely.

StoryPros runs an AI BDR that books 30-plus meetings during a strong week. Seven days of downtime could put more than 30 meeting opportunities at risk.

Odyssey Logistics shows the financial stakes. Its Cognizant and Cognition project reported a 37% net cost saving.

Delivery throughput also increased by roughly one-third. Every code change still required human sign-off.

Let the agent move faster while the system keeps control.

Build the Recall Plan Before You Need It

A model recall shouldn't trigger a week-long rebuild. It should trigger a tested routing change.

Use this checklist:

Inventory

List every workflow and active model.
Record provider, model version, region, and rate limits.
Mark every action that changes customer data.
Identify provider-specific session state.
Document where prompts and tool schemas live.

Prepare fallbacks

Choose one same-provider fallback.
Choose one cross-provider fallback.
Confirm both support the required context and tools.
Store critical state outside provider sessions.
Convert provider responses into one schema.

Test behavior

Build at least 50 fixed contract cases.
Include missing data and tool failures.
Test prompt injection and false claims.
Verify unsubscribe and suppression rules.
Replay the suite against every route.

Control actions

Add idempotency keys to every write action.
Require approval for sensitive changes.
Separate drafting from sending.
Save checkpoints before tool execution.
Block retries after an uncertain write.

Run the drill

Disable the primary route.
Measure time until the fallback takes traffic.
Compare cost, speed, and accepted output rates.
Confirm queued tasks resume once.
Record every manual step.

Run this drill quarterly. Run it again after major prompt, tool, or provider changes.

V1 won't catch every edge case. That's normal.

Your first successful run doesn't prove the system is reliable. Production agents earn trust through repeated tests and boring recoveries.

Frontier models will keep improving. Some will also get delayed, restricted, renamed, repriced, or removed.

Build for that reality.

FAQ

How do I set up fallback routing when an AI model is paused?

Create a routing layer outside the prompt. Send tasks through a primary model, an approved same-provider fallback, a cross-provider fallback, and a human queue.

Store workflow state in your own database. Never depend on one provider's chat session for recovery.

What are contract tests for models?

Contract tests verify that a model follows the fixed rules your workflow requires. Examples include valid JSON, approved claims, tool restrictions, unsubscribe handling, and idempotency keys.

Run them before model changes and at least daily. A failed contract should block the model from receiving live traffic.

How should sales teams write an agent SLA?

Define accepted output rates, tool success, recovery time, duplicate-action limits, and escalation owners. A useful SLA measures completed revenue work, not API uptime.

Set zero tolerance for duplicate sends and unauthorized CRM writes. Route to a tested fallback within a fixed window, such as 15 minutes.

Should I always use the newest frontier model?

No. The newest model may offer better reasoning but worse cost, access, or stability.

Use the cheapest approved model that passes your contract tests. Save frontier models for tasks that truly require them.

Can OpenAI's Agents API replace a recall plan?

No. The Agents API handles sessions, context compaction, tools, subagents, and sandbox choices.

You still need external state, contract tests, routing, action validation, and human escalation. Managed orchestration doesn't provide business continuity.