The CX Foundry method

The simulation and evaluation loop is the heart of it

Conversation data analysis identifies opportunities.

AI + your inputs + Principal expertise creates a working agent.

Working agents are tested against simulated scenarios derived from conversation data.

Evaluation drives changes to the AI agent assets: prompts, tools, knowledge, and scope.

Then simulate and evaluate again. And again. Building from performance, to policy adherence, to infosec and compliance testing in a fast, iterative, AI-native loop.

Simulation and evaluation starts in Stage 1 as soon as there is something worth testing. It becomes the center of Stage 2.

Why it works

It starts from real conversations.

Each stage is justified by working assets from the previous stage.

The evaluation process optimizes for performance and compliance.

It defers a platform decision until the evidence is strong enough to justify it.

The method

Snapshot: find the opportunity

1. Collect a representative sample of contact center recordings, chats, or transcripts. Go broad, and with enough volume to see repeated patterns.
2. Transcribe the conversations and redact private information. The goal is to preserve what happened in the interaction while removing names, account numbers, health details, payment data, or anything the analysis process doesn’t need.
3. Normalize the data into a clean evidence set. Put each conversation into a consistent format with speaker turns, timestamps where available, metadata, call duration, channel, and any known outcome.
4. Analyze every conversation with the approved analysis model. Categorize the conversation, score friction or effort, identify the customer intent, capture the resolution path, and note where the process broke down or required human work.
5. Build clusters of similar conversations. Look for repeated patterns such as “status request,” “appointment reschedule,” “refund follow-up,” or “missing authorization.”
6. Size the opportunity. For each major cluster, estimate volume, average handle time, and automation value. Choose the cluster that is large enough to matter, repeated enough to automate, and simple enough to prove.
7. Infer the process behind the calls. Map what the human agent appears to do: what information they collect, what systems they check, what decisions they make, what policy rules they follow, and where they escalate.
8. Identify the tools the AI agent would need. This usually becomes a short list of backend actions: look up an order, check account status, search availability, verify eligibility, create a case, send a confirmation, or hand off to a person.
9. Build a realistic demo agent and backend stub. Use AI code generation to create a working agent and a realistic fake API so stakeholders can experience the future workflow before production systems are touched.
10. Build a simulated customer agent to test the demo. Have one AI agent play realistic customers against the demo agent, including normal cases and edge cases, then iterate until the agent completes the workflow or hands off safely.
11. Demonstrate the experience to stakeholders. Show the evidence, the opportunity size, the live agent interaction, and the proof from simulated runs so business, operations, and technical teams can align on whether the workflow should move to Stage 1.

The snapshot is complete when the team can say: “Here is a repeated customer conversation pattern, here is why it matters, here is the process behind it, here is the agent we can build, and here is the evidence that it is worth proving properly.” In rare cases, a snapshot might end with: “No such opportunity could be found”, which is just as valuable.

Stage 1: prove the agent works

1. If necessary, gather additional conversation data and repeat the analysis process above.
2. Facilitate prioritization and alignment among stakeholders to identify 2-3 workflows that should be analyzed in more detail. Consider impact, feasibility, and risk.
3. Write opportunity briefs that explain, for each workflow: what customers are trying to do, what the agent should handle, what backend capabilities will be required, and what evidence supports the brief.
4. For each brief, gather the context needed to make the workflow real: SOPs, scripts, policies, escalation rules, API documents, schemas, sample payloads, test data, and business value assumptions. Turn important policy requirements into testable acceptance criteria.
5. Define and build the backend the agent needs. This is usually a server that provides tools like: status, eligibility, search, book, escalate. Use test data so the demo behaves like a real system without needing production access.
6. Build the service agent configuration to leverage the backend tools: role, instructions, boundaries, tone, tool usage rules, escalation rules, and safety requirements. Keep it narrow and voice-friendly. The agent should only promise outcomes its tools are capable of.
7. Build a customer simulation agent based on scenarios seen in the original data. Have it interact with the service agent. Evaluate task completion, policy adherence, tool usage, and escalation.
8. Generate a Stage 1 proof pack that includes all documents, data, analysis, code, config, instructions, simulation and evaluation runs, results, and a data-derived value model and ROI analysis.

Stage 1 is complete when the evidence is clear, the selected workflow is explicit, the backend capability runs, the agent package is active, the agent can complete or safely hand off representative cases, and the proof artifacts support a credible ROI conversation.

Stage 2: secure the launch path

1. Start from the Stage 1 workflows. Use the same evidence, backend, service agent, simulation agent, and proof artifacts as the baseline.
2. Confirm the readiness bar. Define what must be true for performance, policy adherence, customer experience, escalation, infosec, compliance, and business value for the agent to be deployed into production.
3. Expand the simulation set. Add more functional scenarios, harder edge cases, and realistic failure paths based on the original conversations and the Stage 1 proof findings.
4. Add adversarial and red-team testing. Test for identity bypass, social engineering, prompt exfiltration, confidential data leakage, false confirmations, unauthorized actions, unsafe advice, and attempts to push the agent outside its scope.
5. Use evaluation results to improve the agent. Update prompts, tool descriptions, backend behavior, knowledge, escalation rules, and scope. Then rerun the relevant simulations to prove the fix worked.
6. Quantify performance and risk. Report task completion, safe handoff rate, automation potential, policy failure rate, attacker success rate, critical violations, tool errors, and any remaining gaps.
7. Decide how the new experience will integrate into the existing customer journey. Detail the integration points with existing systems and external dependencies that ensure the right contacts reach the new experience, and failed or partially completed interactions are handed off smoothly.
8. Refresh the business case. Use the stronger proof to estimate automatable volume, gross labor saving opportunity, volume, type and handling cost of escalations. Update platform cost assumptions and ROI based on the target platform and operating model.
9. Prepare a reusable simulation and evaluation harness that can be used to validate the production deployment.
10. Create a Stage 2 readiness pack. Include the current agent assets, backend tools, proof runs, transcripts, scorecards, risk and compliance summary, launch-readiness assessment, business value model, and a clear list of what remains out of scope.

Stage 2 is complete when at least one of the Stage 1 workflows has survived broader functional testing, adversarial testing, policy testing, and value analysis; the remaining risks are explicit; and business, operations, technical, security, and compliance stakeholders have enough evidence to decide whether to move toward production. Occasionally, none of the Stage 1 workflows will meet that bar. Again, that is valuable information that avoids a failed deployment and means the organization can refocus its CX transformation efforts.

Production deployment

1. Hand the Stage 2 repository containing agent instructions, config, code and stubbed backend implementation along with the reusable simulation and evaluation harness to the production deployment team.
2. If deploying with the same agentic AI tech as used in the CX Foundry process, simply deploy the Stage 2 assets into a production environment. For other environments, a short ‘translation’ pass is required to refactor the agent instructions, config, and code for the target environment.
3. Replace demo backends with real integrations. Map each test tool to the real API, database, queue, workflow system, or human escalation path. Define authentication, permissions, schemas, retries, idempotency, logging, and failure behavior.
4. Load test data into the backend systems that matches what was used in Stage 2. Re-run simulation and evaluation to validate the deployment.
5. Complete security, compliance, and operational review. Confirm data handling, access controls, secrets management, audit logs, human approval boundaries, retention policies, incident response, and rollback.
6. Define monitoring and support. Track task completion, handoff rate, tool errors, latency, cost, customer satisfaction, policy violations, escalation quality, and model or prompt regressions.
7. Run a staged rollout. Start with internal testing, then shadow mode, then a controlled pilot, then broader production traffic only when live results match the Stage 2 evidence.
8. Convert estimated ROI into measured ROI. Replace assumptions with production data: actual call volume, containment, average handle time impact, labor savings, platform cost, implementation cost, and net value.

Production deployment is complete when the agent is fully deployed, integrated, handling a significant portion of live traffic, and performing close to or beyond the targets established during the simulation and evaluation activities in Stage 2.