Multi-Agent Security Operations Triage

Agents classify, enrich and prioritise incident reports, and a human approves every consequential step.

View code on GitHub

100%priority accuracy on the golden incident set

87.5%category accuracy on the same set

1 retryschema repair before a case goes to a human

0reports dispatched without human approval

The problem

Incident triage is mostly unstructured text and repeated decisions: classify, look up history, set severity, write it up consistently. It suits agents well and suits full autonomy badly, so human approval is a core design constraint here, not an afterthought. I write incident reports in my security work, so I know where the time goes.

What I built

A LangGraph workflow

An LLM classifier, a deterministic enrichment tool, a policy-based prioritiser and an LLM report drafter.

Guardrails

Every output goes through Pydantic schemas with a one-shot repair loop, and emails, phone numbers and NI numbers are redacted before any text reaches a model.

Rules kept out of the model

Severity is model judgement, but which SLA applies is an 8-line policy matrix, because that is a business rule.

How it works

IntakeReport arrives via a FastAPI endpoint; PII redacted
ClassifyCategory, severity and risk as schema-enforced structured output
EnrichSite criticality, related procedures and incident history from a registry
PrioritisePolicy matrix maps severity and risk to P1 to P4 and an SLA
ApproveThe graph interrupts and a human approves before dispatch
EvaluateCost tracking per node; golden-set trajectory evaluation in CI

What I found

Trajectory evaluation on the golden set: 100% priority accuracy and 87.5% category accuracy.
The one category miss is a chemical spill that sits on the safety and environmental boundary, and its priority still lands correctly at P1, because priority never depends on the model's category opinion.
Guardrails are tested with deliberately flaky and hopeless model stubs: schema repair recovers the first and human review catches the second.
Cost is metered per node against a hard per-incident budget.

Limits and honest notes

The golden set is small and the registry is sample data, so this shows the design, not production accuracy.
Human approval is required before dispatch by design, so the system supports a triager and does not replace one.

Skills shown

LangGraphAgent orchestrationPydanticStructured outputsGuardrailsPII redactionCost budgetsTrajectory evaluationFastAPI

Next projectCard Fraud and Disputes Performance Review →