How to Test AI Agents Locally Before Touching AWS (2026)
Test AI agents against LocalStack using fake credentials before granting any AWS access. Run 5 idempotency checks, force 429 and timeout failures, and pass a sandbox IAM gate. Production access comes last.
Test AI Agents Locally Before AWS Pays the Price
AWS says thousands of autonomous agents built its new AWS Emulator. The article describes the guardrails AWS used.
AWS split the work across three agent teams. One wrote tests. One found failures. One fixed them. Every change had to pass a merge gate before release.
Use the same pattern for revenue agents.
Step 1: List Every Action the Agent Can Take
Start with actions, not prompts.
A revenue agent might read an S3 file, write to DynamoDB, send SQS messages, or start Step Functions. It might also update Salesforce or HubSpot.
Write down each action.
| Agent action | Test target | Main risk |
|---|---|---|
| Read campaign files | S3 emulator | Wrong bucket or object |
| Queue enrichment jobs | SQS emulator | Duplicate messages |
| Save prospect state | DynamoDB emulator | Duplicate writes |
| Run outreach workflow | Step Functions Local | Bad retries |
| Fire campaign event | EventBridge emulator | Event loops |
| Read API token | Mock Secrets Manager | Secret exposure |
| Update CRM contact | WireMock or Mockoon | Bad production writes |
Now build a tool allowlist.
```yaml allowed_tools: - s3.get_campaign_file - sqs.queue_enrichment - dynamodb.save_prospect - stepfunctions.start_campaign
blocked_tools: - s3.delete_bucket - secretsmanager.put_secret_value - iam.create_user - crm.delete_contact ```
The model shouldn't have direct AWS SDK access. It should call your tool wrapper.
The wrapper checks the tool name, arguments, account, region, and endpoint. It rejects everything else.
Prompts aren't security controls. A prompt that says "never delete contacts" can't enforce anything.
Agent security starts outside the model.
Step 2: Run AWS and CRM Calls Against Local Fakes
LocalStack is a practical starting point. Its community version is free and covers common AWS APIs.
A basic Docker setup looks like this:
```yaml services: localstack: image: localstack/localstack:latest ports: - "4566:4566" environment: - SERVICES=s3,sqs,dynamodb,stepfunctions,events,secretsmanager - AWS_DEFAULT_REGION=us-east-1 volumes: - "./localstack:/var/lib/localstack" ```
Use fake credentials:
```bash export AWS_ACCESS_KEY_ID=test export AWS_SECRET_ACCESS_KEY=test export AWS_DEFAULT_REGION=us-east-1 export AWS_ENDPOINT_URL=http://localhost:4566 ```
Your SDK should require the local endpoint during tests:
```python import boto3 import os
endpoint = os.environ["AWS_ENDPOINT_URL"]
assert endpoint.startswith("http://localhost:")
sqs = boto3.client( "sqs", endpoint_url=endpoint, region_name="us-east-1" ) ```
For CRM calls, use WireMock or Mockoon. Both have free local tools.
Mock the exact HubSpot or Salesforce responses your agent expects. Include expired tokens, duplicate records, and validation errors.
AWS's emulator uses the same basic pattern. SDK calls go to a local HTTPS endpoint instead of AWS.
There's a catch. An emulator can't fully match production AWS behavior, especially for IAM policies and edge cases.
Use an emulator as your first gate. Test IAM permissions again in a separate AWS sandbox account.
Step 3: Make Duplicate Actions Harmless
Agents retry. Queues retry. Step Functions retries.
If the same call creates three Salesforce contacts, your architecture failed.
Every write needs an idempotency key. Build it from stable inputs.
```python import hashlib
def idempotency_key(account_id, contact_id, campaign_id): raw = f"{account_id}:{contact_id}:{campaign_id}" return hashlib.sha256(raw.encode()).hexdigest() ```
Store that key before taking action:
```python table.put_item( Item={ "idempotency_key": key, "status": "started" }, ConditionExpression="attribute_not_exists(idempotency_key)" ) ```
The condition blocks repeated writes. Your tool wrapper should return the original result instead.
Test it with a simple rule:
1. Send the same tool call five times. 2. Confirm DynamoDB stores one record. 3. Confirm SQS receives one message. 4. Confirm the CRM mock records one update. 5. Confirm all five calls return a safe result.
Idempotency testing is required for revenue agents.
A duplicate analytics event is annoying. A duplicate discount, contract, or outbound sequence can damage trust.
We've built more than 100 automations at StoryPros. The model is rarely the hard part.
State management is.
Step 4: Force Rate Limits, Timeouts, and Bad Retries
Happy-path tests prove little.
Your test harness should return `429`, `500`, and timeout responses on purpose. It should also drop connections halfway through a request.
Use a counter inside your tool stub:
```python attempts = 0
def mock_crm_update(payload): global attempts attempts += 1
if attempts <= 2: return {"status": 429, "retry_after": 2}
return {"status": 200, "id": "mock-contact-123"} ```
Your agent should wait, retry, and stop after a fixed limit. It shouldn't try a different endpoint.
Step Functions supports explicit retry rules:
```json "Retry": [ { "ErrorEquals": ["TooManyRequestsException"], "IntervalSeconds": 2, "BackoffRate": 2, "MaxAttempts": 3 } ] ```
This rule waits two seconds before the first retry, then four seconds before the second. It allows up to three retries.
Also add hard workflow limits:
- Maximum tool calls per run
- Maximum contacts changed per run
- Maximum retry count
- Maximum execution time
- Maximum estimated API cost
- Manual approval for bulk actions
A kill switch that requires a person isn't enough. The workflow must stop on its own when it hits a limit.
Step 5: Gate Production With IAM and CI Tests
Your local test job should have no production credentials.
Not hidden credentials. Not read-only credentials. None.
Block outbound access to AWS and CRM domains from the test container. Allow only localhost and approved mock services.
Add a CI check:
```bash if printenv | grep -E 'PROD|SALESFORCE|HUBSPOT_PRIVATE'; then echo "Production credential detected" exit 1 fi
if [ "$AWS_ENDPOINT_URL" != "http://localhost:4566" ]; then echo "Unsafe AWS endpoint" exit 1 fi ```
Then test the agent in a separate AWS sandbox account. Give it fake contacts and disposable resources.
Use a narrow IAM policy:
```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "sqs:SendMessage", "dynamodb:PutItem", "states:StartExecution" ], "Resource": [ "arn:aws:sqs:us-east-1:111111111111:agent-test-*", "arn:aws:dynamodb:us-east-1:111111111111:table/agent-test-*", "arn:aws:states:us-east-1:111111111111:stateMachine:agent-test-*" ] } ] } ```
Notice what's missing.
No wildcard actions. No production account. No IAM access, Secrets Manager writes, or resource deletion.
Production access comes last. Use a separate role with tighter limits.
StoryPros builds AI agents that take action. That makes testing more important than another prompt workshop.
A chatbot can give a bad answer. An agent can write that answer into 40,000 CRM records.
The Old Lesson AWS Agents Are Teaching Again
This pattern isn't new.
Factories tested machines before workers stood inside them. Payment teams built sandboxes before they allowed live charges.
AI agents need the same discipline.
AWS's emulator announcement got attention because autonomous agents wrote the code. The useful part was the process.
Bounded tasks, repeatable tests, and hard merge gates made the work safer.
Use those controls before an agent gets an AWS role or CRM token.
FAQ
How do I test AWS Step Functions locally?
Run AWS Step Functions Local through its Docker image, or use LocalStack's Step Functions support. Point your AWS SDK at the local endpoint, create the state machine, and test success, timeout, retry, and duplicate-execution paths.
Can LocalStack emulate IAM permissions and policies?
LocalStack can test many AWS calls and some permission behavior. Don't use it as final proof that an IAM policy is safe. Test least-privilege policies in a separate AWS sandbox account before production.
How can I prevent AI agents from accessing production credentials?
Keep production credentials out of local machines and CI test jobs. Use fake keys and local endpoints. Block network access to AWS and CRM domains, then use a sandbox account with narrow IAM roles.
How do I simulate rate limits and retries when testing agents?
Configure a mock endpoint to return `429` responses for the first calls. Confirm the agent follows `Retry-After`, uses exponential backoff, respects `MaxAttempts`, and stops without trying a different endpoint.
What should pass before an AI agent touches production?
The agent should pass tool allowlist, idempotency, timeout, rate-limit, retry, and permission tests. It should also stop automatically when it reaches call, time, record, or cost limits.
Related Reading
How do I test an AI agent against AWS without using real credentials?
Run LocalStack on port 4566 with fake credentials: AWS_ACCESS_KEY_ID=test and AWS_SECRET_ACCESS_KEY=test. Set AWS_ENDPOINT_URL to http://localhost:4566 so every SDK call hits the local emulator instead of AWS. Block outbound access to AWS domains from the test container entirely.
How many times should I send the same tool call to test idempotency?
Send the same tool call 5 times, then confirm DynamoDB stored exactly 1 record and SQS received exactly 1 message. Build the idempotency key from stable inputs using SHA-256, and use a DynamoDB ConditionExpression with attribute_not_exists to block repeated writes.
What AWS permissions should an AI agent have during sandbox testing?
Limit the sandbox IAM policy to sqs:SendMessage, dynamodb:PutItem, and states:StartExecution on agent-test-* resources only. Exclude wildcard actions, IAM access, Secrets Manager writes, and all production account resources. No production access until the agent passes all local tests.