Building Deterministic Infrastructure for Non-Deterministic AI Agents
LLMs are probabilistic, but production infrastructure cannot be.
This talk explores the engineering patterns required to operate AI agents reliably in production from workflow orchestration and retries to state management, failure recovery, tool execution, and human approval gates.
We'll examine how production systems transform unpredictable model behavior into deterministic business workflows and discuss architectural patterns used to build resilient agent platforms.
Key Takeaways:
Understand why LLMs are inherently non-deterministic and why production infrastructure must provide deterministic guarantees.
Learn architectural patterns for building reliable AI agent systems, including orchestration, workflow state management, retries, checkpoints, and failure recovery.
Discover how to safely integrate tools and external systems using validation, permission boundaries, human-in-the-loop approvals, and guardrails.
Explore techniques for making agent workflows observable through tracing, evaluation pipelines, telemetry, and debugging of reasoning chains.
About the speaker
Nishant Gupta:
Nishant Gupta is a Staff Software Engineer at Meta, where he builds large-scale AI infrastructure and distributed systems powering next-generation agentic AI platforms. His work spans distributed computing, inference infrastructure, resource scheduling, production evaluation systems, and reliable AI agents. Prior to his current role, Nishant led infrastructure initiatives managing elastic compute capacity across millions of servers, improving utilization while maintaining strict reliability and performance objectives. His recent work focuses on the operational challenges of deploying AI agents in production, including evaluation frameworks, orchestration, observability, cost optimization, and deterministic infrastructure for non-deterministic AI systems. Nishant is an active researcher with multiple peer-reviewed publications in distributed systems and artificial intelligence, including work on large-scale resource optimization and AI-driven infrastructure. He is also an international conference speaker, having presented at events including AI Engineer World's Fair, DeveloperWeek, IEEE conferences, and other global AI and software engineering conferences. His talks focus on operating agentic AI systems, production evaluation pipelines, distributed inference, and scalable AI infrastructure. Nishant is passionate about bridging cutting-edge AI research with real-world production systems, helping organizations build reliable, scalable, and efficient AI platforms that operate safely at global scale.
About
TestMu Conf
Testμ (TestMu) is the world’s largest virtual conference on agentic engineering and quality, built by the community, for the community. As AI reshapes how we build, test, and ship software, Testμ Conf is where you connect, grow, and lead: agentic workflows, autonomous quality, battle-tested AI playbooks, hands-on workshops, and the engineering culture driving it all.
More Sessions
Join the builders, testers, and innovators shaping the next generation of web experiences.
Testμ Conf 2026 is where they meet.
