At-a-Glance

• The Challenge: Build an AI platform where multiple agents could work together reliably in production

• The Solution: We connected multiple AI agents and models, including Claude, added safety controls, and built a way to continuously check how the system behaves

• Technology: LangGraph, Anthropic API, Claude models, Claude Code, DeepEval, Promptfoo, Langfuse

• Impact: 5,000+ scenarios validated, more reliable routing, safer actions, and measurable AI quality checks

• Differentiator: One system that brings together AI development, safety, monitoring, and continuous validation

Background

 
Phoenix AI is building a platform where several specialized AI agents work together to handle complex user requests.

Instead of using one chatbot for everything, the system decides which AI agent should handle each task, which model should be used, and when tools or external systems should be involved.

The platform uses different AI models for different needs, including Claude models through the Anthropic API.

The challenge was making all of this reliable enough for real production use.

The right agent had to be selected. Context had to stay consistent between handoffs. Tools had to be used safely. And the team needed a clear way to see what the AI was doing when something went wrong.

We helped turn this into a controlled, measurable, and production-ready system.

Goals & Objectives

 
The goal was simple: make the AI system reliable, safe, and easy to improve.

• Reliable Agent Routing: Make sure each request goes to the right AI agent and model
• Use the Right Model for the Right Task: Use Claude and other AI models depending on what each task needs
• Safe Tool Usage: Make sure AI agents only use approved tools and actions
• AI Safety Controls: Reduce prompt injection, unsafe behavior, and unexpected actions
• Continuous Testing: Check the system across thousands of realistic scenarios
• Clear Visibility: See how agents, models, and tools behave and where problems happen
   

Challenges

1. Multiple AI Agents Working Together
Different agents need to hand work to each other without losing context.

2. Choosing the Right Model
Different AI models are better at different tasks, so the system needs to choose carefully.

3. AI Taking Real Actions
Agents can call APIs, tools, and external systems, so mistakes can have real consequences.

4. Security Risks
Prompt injection and too much access can lead to unsafe behavior.

5. Hard-to-See Failures
Traditional logs do not always show why an AI workflow failed.

Partnership & Process

We built the platform across four key areas.

1. Multi-Agent System

Using LangGraph, we created workflows that help different AI agents work together. This included:

• Choosing the right agent
• Passing work between agents
• Keeping context consistent
• Handling multi-step tasks
• Calling external tools
• Managing failures and fallback behavior
   

2. Claude & Multi-Model Integration

Claude models were integrated through the Anthropic API as part of a broader AI model setup. The system can choose Claude or another model depending on the task. This gives Phoenix AI more flexibility and avoids relying on only one AI provider.

Claude Code also supports the engineering team with implementation, debugging, testing, and code review.

3. AI Safety Controls

We added safeguards around how agents behave and what they are allowed to do. This included:

• Prompt injection testing
• Checking inputs and outputs
• Limiting tool permissions
• Giving agents only the access they need
• Protecting sensitive actions
• Safe fallback behavior when the system is uncertain
   

4. Testing & Monitoring

DeepEval and Promptfoo are used to validate the system across more than 5,000 scenarios. These include:

• Real user journeys
• Edge cases
• Long conversations
• Agent handoffs
• Different AI model behavior
• High-risk situations
• Prompt injection and other adversarial scenarios

Langfuse gives the team a clear view of:

• Which agent was selected
• Which model was used
• Prompts and responses
• Tool usage
• Failures
• Workflow success

Explore Our QA Services 

Outcomes & Impact

More Reliable AI Agent Workflows
Different AI agents can work together through clear routing and handoff rules.

Claude Integrated Into Real Workflows
Claude models are used directly inside the multi-agent system based on task needs.

Flexible AI Model Usage
The platform can use different AI models instead of depending on a single provider.

5,000+ Scenarios Validated
The system is tested across real-world, edge-case, and high-risk situations.

Safer AI Actions
Permissions and safety controls limit what agents can do.

Better Visibility
The team can see how agents, models, prompts, and tools behave across the system.

Continuous Quality Checks
Changes to models, prompts, agents, and workflows can be checked before they reach production.
 

Looking Ahead

Phoenix AI now has a strong foundation for adding more agents, models, tools, and automated workflows while keeping the system safe and under control.

Building AI agents is easy. Making them reliable in production is the hard part.

 
What can we build together?