Ship Faster. Test SmarterJoin Now

Agenda

SHOWING 15 SESSIONS

05:00 - 05:15 AM (PT)
15 MINS
Welcome Note
05:45 - 06:15 AM (PT)
30 MINS
Building Deterministic Infrastructure for Non-Deterministic AI Agents
SESSION
06:15 - 07:00 AM (PT)
45 MINS
Context That Dreams
SESSION
Every organization is racing to build "the context layer": pipe Teams, Zoom, Outlook, Jira, and Confluence into one indexed knowledge store, so that every AI agent can retrieve the right snippet at runtime. That's the RAG playbook, and it works — until an agent asks a question whose answer was never written down anywhere. Retrieval can only return what was captured. If the context doesn't exist as data, the agent runs blind — and quietly, that ceiling caps every AI workflow you ship on top of it.
This talk proposes a different contract for the context layer: one that behaves less like a filing cabinet and more like a brain — simulated, not literal. Instead of sitting idle between queries, the layer keeps working. Every captured fragment becomes a node in a knowledge graph — topics, people, decisions, risks — and a continuous consolidation loop keeps revisiting those nodes, testing how they correlate across meetings, tools, and time. When disconnected fragments keep pulling toward each other, the layer mints something genuinely new: an emergent concept — an idea that exists in no document, born purely from the connections. And because every emergent idea carries full lineage back to its source conversations, synthesized knowledge stays auditable rather than hallucinated.
VIEW FULL SESSION
Jaydeep Chakrabarty
Senior Director - AI, Piramal Finance
06:15 - 07:00 AM (PT)
45 MINS
From AI Assistants to AI Coworkers: How Engineering Teams Ship Faster with Trusted Enterprise Context
SESSION
Generic AI assistants can answer questions, but they don't understand your codebase, your incidents, or how your team actually works. That gap is what separates a helpful chatbot from a genuine engineering coworker. This session explores the shift from general-purpose assistants to AI systems that reason over the full context of software engineering: code, tickets, documentation, incidents, and the organizational knowledge that lives between them.
At the heart of this evolution is enterprise context, the foundation for AI that can meaningfully support modern engineering teams. This talk unpacks why that context matters so much, and shares practical lessons on building AI that developers actually trust, including how to balance automation with human oversight and keep engineers in control as systems take on more of the work.
VIEW FULL SESSION
Nilesh Dalvi
Engineering Leader, Glean
07:00 - 07:45 AM (PT)
45 MINS
Architects of Trust: Redefining Quality Leadership in an Agentic World
SESSION
When software evolves from predictable, deterministic scripts to autonomous, non-deterministic agents, classical quality metrics crumble. A pristine report showing 94% test coverage can easily mask a silent, recursive prompt loop that costs $400,000 in production before a single alert fires. In this high-stakes agentic era, the ultimate currency of engineering is no longer test coverage—it is trust.
True quality leadership requires balancing the inherent chaos of autonomous AI systems with the rigid demands of compliance, safety, and business risk. Drawing from the front lines of shipping AI at scale, this session will deliver a pragmatic, battle-tested playbook for technology leaders, unpacking an Empirical Trust Framework that treats trust as a hard metric, mapping statistical risk boundaries directly to corporate SLAs.
VIEW FULL SESSION
Sobhitha Neelanath
Senior Manager, Software Engineering, Salesforce
07:45 - 08:30 AM (PT)
45 MINS
Testing the Future: Time-Travel Simulation for Financial Systems
SESSION
Financial platforms must validate lifecycle events such as interest accrual, scheduled and partial payments, delinquency, charge-offs, refunds, and settlements before those events naturally occur. Waiting weeks or months is not practical when products and partner integrations must launch quickly and confidently.
This session explores how a lifecycle-aware Test Data Management Portal can create production-like financial states and programmatically advance accounts and loans through future servicing events. Attendees will learn how deterministic time-travel simulation enables repeatable end-to-end validation across servicing, payments, applications, APIs, databases, and customer communications. The session will also cover the architectural principles, governance considerations, and lessons learned from implementing this capability at enterprise scale.
VIEW FULL SESSION
Rakesh Sukla
Director of Engineering - Platform and Quality Engineering, Bread Financial
07:45 - 08:30 AM (PT)
45 MINS
Can You Trust Your ChatBot: Techniques for Testing LLM Responses
SESSION
08:30 - 09:00 AM (PT)
30 MINS
The Right Model for the Right Job: Cutting LLM Costs Without Cutting Quality
SESSION
Most AI agents never make it past the demo stage, and the gap between a prototype and a production system comes down to evaluation. I cover how to choose between open source, proprietary, and fine-tuned models, define custom metrics for your use case, calibrate LLM judges for cost-effective assessment, and track results over time to measure real progress. Attendees leave with practical tools to move from proof-of-concept to production.
VIEW FULL SESSION
Viktoria Semaan
Principal Technical Evangelist, Databricks
08:30 - 09:00 AM (PT)
30 MINS
System Prompt Design for Voice Agents
SESSION
In this session, we'll explore how system prompt design shapes real-time voice AI behavior, from pacing and hesitation handling to confirmations, interruptions, tool calling, and recovery. Using a live Vapi demo, we'll compare approaches to system prompt design across the same use case to understand the implications of small changes. We will also discuss how model selection affects how your voice agent interprets and uses your system prompt. Attendees will leave with a practical framework for designing voice agents that can listen naturally, use tools reliably, and feel more patient, capable, and human in conversation.
VIEW FULL SESSION
Amanda Martin
Head of Developer Relations, Vapi
09:45 - 10:15 AM (PT)
30 MINS
Keynote Session by Francesca Lazzeri
KEYNOTE
10:15 - 11:00 AM (PT)
45 MINS
Build Trustworthy AI Agents powered by Evals
SESSION
Agentic AI has captured the industry's attention. As more people experiment with LLMs, they develop intuition aka vibes around whether models perform well. While math and classification are easy to verify, generative tasks aren't. Adding agents makes evaluation far more complex. This talk highlights key challenges in productionizing agentic applications: non-deterministic outputs, elusive ground truth, and models that mysteriously regress. Yet 'LGTM' isn't a deployment strategy.
Unlike chatbots that resemble sophisticated search engines, we expect agents to actually perform tasks; making security, bias, and privacy non-negotiable before unlocking real use cases in healthcare, finance, and legal. We'll explore why agent evaluation is fundamentally harder than traditional ML testing: multi-step reasoning chains, tool-use side effects, and more. Topics include building evaluation datasets reflecting production scenarios, automated LLM-as-judge pipelines, knowing when human-in-the-loop is unavoidable, and detecting regressions before users do.
VIEW FULL SESSION
Rushabh Mehta
Software Engineer, Meta
10:15 - 11:00 AM (PT)
45 MINS
Real-world Lessons from Scaling Test Automation at Microsoft
SESSION
As software systems grow in scale and complexity, traditional approaches to test automation are no longer enough. At Microsoft, we are evolving test automation by combining scalable automation platforms with AI agents that help engineers generate, maintain, and improve tests more efficiently.
In this session, I will share real-world lessons from scaling test automation across large engineering teams. We will explore how standardized automation frameworks, reliable CI/CD integration, and self-service tooling enable quality at scale. I will also demonstrate how AI agents are transforming the developer experience by generating Playwright tests from requirements, creating meaningful test cases, suggesting improvements to existing tests, and reducing the maintenance effort required to keep automation reliable.
VIEW FULL SESSION
Deepak Kamboj
Senior Software Engineer, Microsoft
10:15 - 11:00 AM (PT)
45 MINS
Canal-mania and the Philosopher's Stone
SESSION
We think the AI revolution is nearly built. It's barely begun. Drawing on the industrial revolution — its manias, its missing parts, and the tinkerers who actually built it — Magnus makes the case that we're still digging canals, and shows what's left to build and who gets to build it.
We are very early, and there is still a lot to build: we are in the stage where many of the parts and practices still need to be invented. This talk focuses in particular on the need for tools to collaborate with AI on new abstractions, and to create more autonomous systems that still keep humans in the loop for guidance and verification — with real experiences from building this at Sanity.
VIEW FULL SESSION
Magnus K. Hillestad
CEO & Co-founder, Sanity
11:45 AM - 12:30 PM (PT)
45 MINS
Backwards Scoring: Ranking Test Suites by Which Real Incidents They Would Have Caught
SESSION
Test coverage is the metric everyone has and nobody trusts. A 90%-covered codebase still ships incidents. The question worth answering is harder: of the tests we run, which ones have ever caught anything that mattered?
This paper proposes a backwards effectiveness score. For each historical incident in our streaming QE database, we replay candidate test suites against the pre-incident state of the system and record which tests would have failed. Tests that fail on real historical incidents score high. Tests that have never failed on anything real score low, regardless of code coverage or assertion count.
VIEW FULL SESSION
Partha Sarathi Samal
Quality Engineering Manager, Paramount
12:30 - 12:45 PM (PT)
15 MINS
Closing Note

Join the builders, testers, and innovators shaping the next generation of web experiences.
Testμ Conf 2026 is where they meet.