Abstract
AI is changing our systems from two directions simultaneously - and most teams are only watching one.
From the outside, AI-powered consumers interact with platforms in ways humans never did: relentless parallel requests, creative exploitation of undocumented behaviors, and usage patterns that make traditional load tests irrelevant. Your stress tests assumed human patience. AI has none.
From the inside, AI is now embedded in development workflows - writing code, finding bugs, generating fixes. Engineers are shipping faster, but the failure modes are new: sycophantic overconfidence in AI outputs, quantity-over-quality optimization, dissolved ownership of generated artifacts, and review pipelines drowning under machine-rate output.
This talk explores the guardrails required when AI operates on both sides of your platform, organized across three dimensions:
- People - How trust calibration evolves when AI is your collaborator. We'll share findings on how engineers relate to AI-generated work: the ones who trust it too much, the ones who reject it entirely, and the behavioral shifts that actually improved outcomes.
- Process - How development workflows change when output exceeds human review capacity. We'll cover blind validation techniques, two-stage sign-off gates (where AI validates against evidence before SMEs reconcile), and the process redesigns that prevent reviewer burnout while maintaining quality bars.
- Technology - How to improve AI confidence at scale using structured knowledge bases, multi-model disagreement detection (not consensus - disagreement is the signal), evidence-grounded scoring across four dimensions, explicit abstention when confidence is low, and staleness-aware systems that know when their own knowledge is out of date.
The through-line: traditional engineering assumed a human author and a human user. When AI is both, every assumption about quality, resilience, and trust needs to be re-examined - and the guardrails you need aren't just technical. They're psychological and procedural too.
Speaker
Daria Barteneva
Principal Site Reliability Engineer in Observability Engineering @Microsoft Azure
Daria is a Principal Site Reliability Engineer in Observability Engineering in Azure. With a background in Applied Mathematics, Artificial Intelligence, and Music, Daria is passionate about machine learning, diversity in tech, and opera. In her current role, Daria is focused on changing organisational culture, processes, and platforms to improve service reliability and on-call experience. She has spoken at conferences on various aspects of reliability and human factors that play a key role in engineering practices, and has written for O'Reilly.
Speaker
Tim Ren
Partner Group Engineering Manager @Microsoft
Tim is a Partner Group Engineering Manager at Microsoft, leading the engineering teams behind data collection, edge, metrics, and infrastructure for the company's observability platforms. Tim is passionate about telemetry at scale, the evolving relationship between AI and engineering work, and chamber music. In his current role, he's focused on the convergence of observability and AI — both as a tool reshaping how engineering teams ship, and as a new class of consumer reshaping the systems they touch. Tim is based in the Pacific Northwest, where he runs a home lab, enjoys violin, and is slowly losing a battle against his lawn.
