How to Build a QA Gate for Your AI BDR (2026 Guide)
AI BDR tools send unchecked messages. DoorDash hit 100% QA coverage across 19,000 agents using Observe.AI and AWS. Put a separate AI checker before your send action. Score every message for compliance (40pts), relevance (35pts), and deliverability (25pts). Block anything under 75.
Your AI BDR Needs a QA Gate
Most AI BDR software is mail merge with an LLM attached.
Automated QA for AI agents means checking every message before it sends. The gate scores compliance, relevance, and deliverability risk. Failed messages get blocked, fixed, or sent to a human.
DoorDash now evaluates nearly 100% of support interactions across 19,000 agents. Your outbound agent shouldn't send unchecked emails to 500 prospects.
Step 1: Copy DoorDash's QA Model
DoorDash had a math problem.
Its QA team couldn't manually review enough support interactions across 19,000 agents. Small samples missed issues until they became large problems.
DoorDash worked with Observe.AI and Amazon Web Services. AWS provided transcription and AI infrastructure. Observe.AI evaluated nearly every customer interaction.
The Observe.AI DoorDash case study reports three major changes:
- QA coverage reached nearly 100%.
- Product issues surfaced in near real time.
- Human reviewers moved from scoring calls to investigating problems.
DoorDash could spot friction around DashPass and new product launches almost immediately. Those signals had taken days or weeks to find.
That model also works for outbound sales.
Most sales teams review a few AI-generated emails after they send. That's an autopsy, not QA.
Your checker must sit before the send action. Every email, LinkedIn message, and SMS passes through it.
Walter Shewhart introduced statistical quality control at Bell Labs during the 1920s. Sampling made sense when checking every physical item was expensive.
AI changed the cost.
A second model can inspect every outbound message in seconds. Sampling 2% of messages is now a choice, not a limit.
Tools and price: Observe.AI uses custom pricing. AWS charges based on service usage. Request current quotes before choosing either platform.
Expected outcome: Every outbound message receives a recorded QA decision before it reaches a prospect.
Step 2: Put the Checker Before the Send Button
Your AI BDR should never have direct permission to send messages.
It should draft the message. A separate checker should decide whether that draft can leave your system.
Use this flow:
> CRM or prospect database → AI BDR writer → deterministic checks → AI checker → approval router → messaging provider → audit log
A DoorDash-style version adds Observe.AI and AWS:
> CRM → AI writer → n8n → AWS data and model services → Observe.AI evaluation layer → approval gate → SendGrid, HubSpot, Outreach, or Twilio
Observe.AI's DoorDash work focused on customer support conversations. Don't treat it as an outbound email checker.
The lesson is the setup. Keep writing separate from evaluation. Check every interaction.
At StoryPros, we use n8n for this kind of workflow. It gives you clear routing, retries, database writes, and human approval steps.
A 2026 study of more than 6,000 public n8n workflows found that fallbacks, repair loops, alerts, and approval gates were still uncommon.
That's the problem.
Connecting Claude or OpenAI to HubSpot takes an afternoon. Building failure paths takes real work.
Your gate needs five parts:
1. A versioned policy file. 2. Deterministic validators. 3. An independent model evaluator. 4. Human approval for risky messages. 5. An audit record for every decision.
Never let the writing model grade itself. That's like letting a salesperson approve their own expense report.
Tools and price: n8n has a self-hosted option and paid cloud plans. Model APIs and AWS services charge by usage. Check current rates because model pricing changes often.
Expected outcome: No message can bypass QA, even when the BDR workflow retries or loops.
Step 3: Score Compliance, Relevance, and Risk
A useful checker needs a written rubric.
"Does this email look good?" isn't a rubric. It's a vague prompt wearing a tie.
Use three scoring groups.
| Category | Weight | What gets checked |
|---|---|---|
| Compliance | 40 points | Consent, suppression status, claims, required identity, unsubscribe language |
| Relevance | 35 points | Correct person, company facts, role fit, trigger evidence, offer fit |
| Deliverability | 25 points | Spam phrases, links, length, formatting, repetition, domain risk |
Set explicit outcomes:
- 90–100: Send automatically.
- 75–89: Rewrite once, then score again.
- Below 75: Block and send to human review.
- Any hard-rule failure: Block regardless of total score.
These are starting thresholds. Adjust them based on replies, complaints, and blocked messages.
Some rules shouldn't use an LLM.
Use deterministic checks for email syntax, suppression lists, duplicate contacts, missing unsubscribe text, banned domains, and link counts. A model shouldn't "reason" about whether an address appears on your suppression list.
Use the AI evaluator for context.
It should compare each claim with CRM fields and approved research. It should flag unsupported statements such as "I saw your team is hiring" when no source confirms it.
A clean evaluator output looks like this:
- Decision: allow, rewrite, or block
- Compliance score: 0–40
- Relevance score: 0–35
- Deliverability score: 0–25
- Failed rules: rule IDs
- Evidence: CRM fields or source URLs
- Required fix: plain-language instruction
- Policy version: exact version number
This is where most "AI hallucination" debates fall apart.
The model didn't wake up confused. Your system let an unsupported claim pass without evidence or validation.
Tools and price: Regex, database checks, and suppression queries add no software fee inside your workflow. Model evaluation creates a per-message API cost.
Expected outcome: Every send decision is consistent, explainable, and easy to audit.
Step 4: Write Policies the Agent Can't Debate
Your QA policy must separate hard blocks from judgment calls.
Hard blocks are binary. The message sends or it doesn't.
Start with rules like these:
- Block contacts on any suppression list.
- Block addresses with invalid syntax.
- Block claims without a matching source field.
- Block fake reply prefixes like "Re:" on first contact.
- Block hidden tracking links when policy forbids them.
- Block messages missing the sender's identity.
- Block duplicate outreach inside your chosen cooldown period.
- Block legal, medical, or financial claims without approval.
- Block messages containing protected personal data.
- Block any tool call outside the approved provider list.
Then add model-scored rules:
- Does the opening sentence match the recipient's role?
- Is the cited trigger within your allowed age?
- Does the offer solve a problem tied to that trigger?
- Could the message apply to 1,000 unrelated people?
- Does the tone create trust or signal mass automation?
Generic personalization should fail.
"Love what you're building at Acme" isn't relevant. It says nothing.
The Open Agent Passport paper makes the same point for agent actions. Agents need permission checks before tool calls, not reviews after damage occurs.
In the paper's live tests, social engineering beat a permissive model 74.6% of the time. A restrictive policy blocked all 879 comparable attempts.
Model screening and deterministic authorization work together.
Send routine messages automatically. Route borderline claims to a person. Block forbidden actions without discussion.
Tools and price: A policy file can live in GitHub, AWS, or your workflow database. Human review costs depend on review time, so track minutes per escalated message.
Expected outcome: Your AI governance for BDR becomes an enforceable policy, not advice buried in a prompt.
Step 5: Reach 100% QA Coverage in Stages
Don't start by sending 10,000 messages.
V1 usually reaches 60% to 70% of your target performance. That's normal. The first launch gives you failure data.
Begin in shadow mode.
The checker scores every message, but humans still control sending. Compare the checker's decisions with reviewer decisions.
Track these numbers:
- Percentage of messages scored
- Percentage blocked
- Percentage rewritten
- Human override rate
- False approval rate
- Unsupported claim rate
- Duplicate contact rate
- Spam complaint rate
- Bounce rate
- Positive reply rate
- Meetings booked
- Cost per approved message
- Review minutes per escalation
Coverage should be simple:
QA coverage = messages checked ÷ messages prepared
If your agent prepares 1,000 messages and checks 940, coverage is 94%. The missing 60 messages are the risk.
Move to automatic sending only after every route hits the checker. Test retries, timeouts, bulk imports, manual sends, and provider failures.
DoorDash's biggest gain was coverage. Nearly 100% coverage exposed patterns that small samples missed.
Medallia reported a similar result in July 2026. Its Smart Topic Builder made topic discovery five times faster and model auditing six times faster.
Without QA, automation makes mistakes faster.
Calculate your return using actual operating data:
Monthly QA value = avoided loss + reviewer time saved + added revenue − QA costs
Avoided loss can include wasted sends, damaged domains, refunds, legal review, and lost pipeline. Calculate added revenue from tracked meetings and closed deals, not vendor promises.
StoryPros builds AI agents that take action. Our best AI BDR books more than 30 meetings per week.
That output matters only if the system protects trust. Spam at scale isn't sales automation.
Tools and price: Keep model, AWS, n8n, review labor, and messaging fees separate. That shows which layer adds cost.
Expected outcome: Every message gets checked, risky messages stop, and humans investigate patterns instead of reading random samples.
FAQ
Does DoorDash use AI for customer service?
Yes. DoorDash uses Observe.AI with AWS services to evaluate nearly 100% of customer support interactions across about 19,000 agents. The system tracks sentiment, comprehension, behavior, and emerging customer issues.
Does DoorDash run on AWS?
AWS supports the DoorDash and Observe.AI quality-monitoring system through transcription and AI infrastructure. The cited announcement doesn't prove that every DoorDash product runs entirely on AWS.
How much does Observe.AI cost?
Observe.AI doesn't publish a simple per-seat price in the cited material. Pricing requires a custom quote based on interaction volume, features, integrations, and support needs.
What is automated QA for AI agents?
Automated QA for AI agents checks each proposed action against rules before execution. For an AI BDR, that means scoring every message for compliance, relevance, and deliverability before sending.
Does 100% QA coverage require humans to read every message?
No. Every message should pass automated checks, while humans review only risky or uncertain cases. DoorDash used automation to move human QA teams away from manual scoring and toward root-cause analysis.
Related Reading
What score does an AI BDR message need to send automatically?
Messages scoring 90 to 100 points send automatically. Scores of 75 to 89 trigger one rewrite attempt before re-scoring. Anything below 75 goes to human review and is blocked from sending.
How did DoorDash get to 100% QA coverage?
DoorDash partnered with Observe.AI and AWS to evaluate nearly 100% of support interactions across 19,000 agents. Before that, manual sampling missed problems until they grew large. The system moved human reviewers from scoring calls to investigating root causes.
How do you calculate QA coverage for an AI BDR?
Divide messages checked by messages prepared. If your agent prepares 1,000 messages and checks 940, coverage is 94%. The 60 unchecked messages are your risk exposure.