From SRE to FDE: SLIs for AI Systems
AI systems fail differently: slow drift instead of hard down. SRE discipline ports directly across: define service-level indicators for groundedness, latency and cost, then page on burn rate like any other service.
The problem in prod
- Silent quality decay as documents, user phrasing and models drift apart.
- P99 latency spikes from rerankers and long contexts that averages hide.
- Nobody knows the cost per resolved task, so a usage surge becomes a budget incident.
Loop engineering
Track grounded-answer rate on the golden set plus live sampled thumbs signals. Alert on 7-day drift, review weekly with the customer team.
Separate time-to-first-token from full-completion time. Users forgive slow finishes, never slow starts. NTT DATA proved fast incident loops build trust.
Budget per resolved task with per-team quotas and auto-throttle. Report cost next to quality in the same dashboard.
Production hardening checklist
- Canary model and prompt changes with automatic rollback on SLI burn.
- Runbooks for the top five AI failure modes, rehearsed with the client team.
- On-call rotation that includes a customer-side owner, not just vendor staff.
- Postmortems feed new eval cases, closing the loop back into the harness.
Deploying this in MY
For MY rollouts, add PDPA breach-notification timing (72 hours) to the incident runbook and keep log retention windows aligned with BNM expectations before the first incident, not after.
Malaysia FDE guide →