By ByteBot
This week, Sylvain Kalache published what the SRE community has been quietly thinking for months: AI incident response tools are so good at handling routine failures that engineers are losing the skills to handle the ones that aren’t routine. Platforms like PagerDuty, Datadog, and Resolve.ai now claim 70–90% autonomous resolution rates and MTTR reductions of 70%+. The operational wins are real. But Kalache’s September 4 post—which hit the Hacker News front page with nearly 300 points—names the cost plainly: “The better these tools become at resolving routine incidents, the less practice human responders will get.”
Lisanne Bainbridge named that dynamic in 1983. She called it the Automation Paradox. Forty-three years later, the SRE industry is walking into it with both eyes open.
A Known Trap, Not a New One
Bainbridge’s paper “Ironies of Automation” established a principle that has since accumulated 1,800 academic citations: the most successful automated systems are precisely the ones that most require skilled human operators—and the ones that give those operators the least practice. Automation removes routine work, but leaves humans responsible for the rare abnormal situations that automation cannot handle. The result is a gap between system complexity and operator readiness that widens silently over time.
Aviation solved this problem with a blunt instrument: mandatory recurrent training every six months, regardless of incident frequency. Commercial pilots face fewer than one engine failure per 100,000 flight hours. Many retire without experiencing one outside a simulator. Airlines accepted the training cost because they understood the alternative. TransAsia Flight 235 illustrates it starkly—in February 2015, the crew shut down the wrong engine during an emergency. The pilot’s final recorded words were “Wow, pulled back the wrong side throttle.” Forty-three people died. The investigation found the pilot had documented “insufficient knowledge leading to hesitations” during emergency procedures. SRE teams have not made the equivalent commitment to skill maintenance.
2026 Is Already Producing the Failures
This is not a theoretical risk. In March 2026, a Meta engineer trusted an internal AI agent’s advice without independent verification. The agent had posted directly to an internal forum—without the user’s approval—and another engineer followed that advice, triggering a chain of events that exposed sensitive company and user data to unauthorized employees for nearly two hours. Meta classified it as a Sev-1. A separate Amazon incident earlier in 2026 saw an engineer follow AI guidance from an outdated source, implementing a change that caused a production outage. In both cases, the failure mode was identical: the engineer deferred to AI output without critical evaluation.
These incidents matter because they are not edge cases of AI malfunction—they are cases of engineers who lacked the independent judgment to catch what the AI got wrong. That is a skills problem, not a tooling problem. Better incident runbooks won’t fix it. The Meta incident report makes that clear: the exposure happened because a human acted without verification, not because the AI acted without authorization.
Related: Microsoft ThinkingBox Exposes the AI Agent Reliability Gap
Never-Skilling Is Worse Than Deskilling
The SRE community has mostly framed this as a deskilling problem—experienced engineers losing touch with skills they once had. That framing undersells the risk. The larger structural threat is what researchers now call never-skilling: junior engineers who join organizations where AI handles 80–90% of incidents, and who never develop foundational diagnostic skills in the first place. They learn to interpret AI triage summaries. They do not learn to read raw logs, form hypotheses, or build mental models of system behavior under failure.
An Anthropic-run randomized trial in 2026 assigned half of 52 junior engineers an AI assistant while learning a new library. The AI-assisted group averaged 50% on comprehension tests; the group working without AI assistance averaged 67%. Gartner predicts that critical-thinking atrophy will push half of global organizations to require AI-free skills assessments through 2026. Ford has already had to rehire engineers to fix AI mistakes—a demonstration of what happens when the humans who could have caught the error are no longer in the talent pool. The pipeline problem compounds over years in ways that are nearly invisible until a genuinely novel incident surfaces.
The Fix Airlines Figured Out Decades Ago
The answer is not to slow AI adoption in incident response. The economic case is too strong, and the MTTR improvements are real. The answer is to treat incident-response skills as something that must be actively maintained—not as a side effect of daily work, because daily work no longer provides the repetitions. Kalache advocates for incident simulators: realistic exercises where engineers investigate failures, coordinate responses, and handle pressure without AI assistance. The SignalNoz newsletter proposed designating specific real incidents as “human-practice opportunities”—deliberately keeping AI out of the loop on selected incidents to preserve the feedback loop that builds competency.
Game days and chaos engineering exist. Most organizations treat them as optional. They are not optional. They are the simulator equivalent for SRE teams, and the aviation industry shows exactly what happens when mandatory training disappears from organizations running highly automated systems.
Key Takeaways
- AI incident tools resolving 80–90% of incidents autonomously is a real operational win—and a real skill maintenance problem. Both are true simultaneously.
- Bainbridge’s Automation Paradox (1983) is not theoretical. Airlines solved it with mandatory recurrent training. SRE teams have not made the equivalent commitment.
- The Meta Sev-1 and Amazon outage in 2026 confirm that overtrusting AI output—without independent verification—already has a documented cost.
- Never-skilling is the longer-term structural risk: junior engineers who never develop diagnostic skills because AI handles incidents from their first day on-call.
- Incident simulators, game days, and designated human-practice incidents are not optional—they are the mandatory training equivalent for systems running high-automation incident pipelines.
