Multi-Agent Security Operations Triage
Incoming incident reports get classified, enriched with context, prioritised and drafted into structured reports by cooperating agents, with a human approving every consequential step. Not an autonomous toy: an agent system built the way you would have to build it for a real operations team.
Why this project
Incident triage is work I know from the inside. In my security and compliance roles I have written and processed the exact kind of unstructured incident reports this system handles, and I know where the time goes: classification, looking up related history, deciding severity, and writing it all up consistently. It is a genuinely good fit for agents, and a genuinely bad fit for full autonomy, which is why human-in-the-loop approval is a core design constraint rather than an afterthought.
What it does
- Orchestrates a LangGraph workflow: an LLM classifier, a deterministic enrichment tool that pulls site criticality, procedures and incident history from a registry, a policy-based prioritiser, and an LLM report drafter
- Forces every agent output through Pydantic schemas with a one-shot repair loop: a response that fails validation gets one retry with the error attached, then routes to human review instead of guessing
- Redacts emails, phone numbers and NI numbers before any text reaches a model
- Keeps priority out of the LLM's hands entirely: severity assessment is model judgement, but "which SLA applies" is an 8-line policy matrix, because that is a business rule
- Meters cost per node against a hard per-incident budget, and interrupts before dispatch so a human approves every report (LangGraph checkpointing)
- Evaluates full agent trajectories against a golden incident set in CI, not just final answers
Architecture
Results
- Trajectory evaluation on the golden incident set: 100% priority accuracy, 87.5% category accuracy
- The one category miss is a chemical spill that genuinely sits on the safety/environmental boundary, and its priority still lands correctly at P1, because priority never depends on the LLM's category opinion. That separation is the design working as intended.
- Guardrails are tested with deliberately flaky and hopeless LLM stubs; schema repair recovers the first, human review catches the second
Skills demonstrated
- Agent orchestration with LangGraph: state machines, conditional routing, checkpointing, HITL interrupts
- Structured outputs and schema-driven agent design with Pydantic
- Production agent concerns: guardrails, PII redaction, cost budgets
- Agent evaluation beyond vibes: golden-set trajectory testing in CI
- Domain knowledge from working real incident and compliance workflows