AI Agent security evaluation
automated auditing inside a PSA platform
Strategic intelligence and risk advisory
+ 200-scenario evaluation suite, wired into
CI/CD
At-a-Glance
Forward Global's PSA agent read client documents and acted on them: writing to client records, opening billing, drafting outbound communications. It would follow instructions hidden inside those documents as readily as instructions from its own operators. A single planted file could have moved real work through the firm's systems.
The exposure wasn't in the model. It was in the plumbing around it. We mapped it, ranked it, and left behind an automatic evaluation that now runs on every build.
Background
Agentic AI workflow built into the PSA platform, acting across back-end systems on the firm's behalf.
Agentic AI workflow built into the PSA platform, acting across back-end systems on the firm's behalf.
Produced good output, but had never been tested adversarially.
Nothing re-checked it after a model, prompt, or data change.
Scope
The PSA AI agent, tested against its real configuration, permissions, and tool wiring. Production untouched.
Assessed the system around the model (orchestration, tools, memory, data flow, governance), not the foundation model itself.
Synthetic and sanitized data only.
Engagement & Process
Solution audit
Mapped the agent end to end: input entry points, callable tools, identity permissions, memory reads and writes. Most findings came from here; none were about the model.
Red-teaming
Live attacks: indirect injection, multi-turn escalation, memory poisoning, tool-call manipulation. Each finding re-run several times before it counted.
Evaluation suite
200 tests built with Promptfoo and DeepEval: injection refusal, data-exfiltration blocking, tool-call validation, PII handling, retrieval faithfulness.
AIVSS scoring
CVSS severity + AARS agentic amplification, used to rank where risk concentrated so remediation hit the right components first.
Engagement & Process
↓20%
Production incidents
↑40%
Agentic features delivery
↑90%
Security vulnerabilities fixed
What We Found
Risk concentrated in the ingestion path and the tool layer, not the model. Around 80% of confirmed findings traced to those two layers. (Illustrative; replace with real.)
Indirect prompt injection (top priority).
The agent acted on instructions hidden in client input as if they were its own. With write and billing access, a manipulated document could become a real action.
Over-broad tool permissions.
The agent could write far beyond its tasks; combined with injection, that's what made the above possible.
Weak memory isolation.
A note planted in one context surfaced in another and persisted as if it were policy.
No tool-call validation.
Malformed calls passed through unchecked.
PII retrievable from memory.
Client PII searchable in the memory index with no access check.
Thin traceability.
No way to tie an agent action back to the input that caused it.
What We Delivered
Executive risk summary, organized by where the risk sits.
Technical findings report with reproduction steps.
Recommendations ranked by risk and effort.
Target-state guidance for the ingestion, tool-access, and memory layers.
200 tests + failing checks wired into CI/CD, re-testing on every model, prompt, or data change. Remediation stayed with Forward Global's engineers.
Outcomes & Impact
Closed several critical gaps: The injection-driven tool path, the over-broad permissions behind it, the memory-isolation weakness.
Closed several critical gaps: The injection-driven tool path, the over-broad permissions behind it, the memory-isolation weakness.
Thousands of automatic agentic evaluations run across builds, each re-checking the full scenario set.
“EnGenious showed us exactly where the agent was exposed, and the checks now run on every build, so we ship new features faster without reopening old risks.”
Forward Global
LOOKING AHEAD
If you run an agent with tool access, your risk surface isn't the model. It's what the agent is allowed to touch and how much it trusts what it reads. Forward Global went from not knowing where that surface was to a ranked map of it, plus an evaluation that holds the line on every build.
Your agents are already taking actions. The question is whether you can see where they're exposed, and prove it stays closed on every build. Let's redefine what security assurance can look like for your team.