Agenda

AI is changing the role of testing from finding defects to building trust. This session will explore how Quality Engineering must evolve to assure AI-enabled systems across reliability, security, governance, observability, and real-world outcomes.

LLMs are probabilistic, but production infrastructure cannot be.

This talk explores the engineering patterns required to operate AI agents reliably in production from workflow orchestration and retries to state management, failure recovery, tool execution, and human approval gates.

An AI agent is exactly as confident when it's right as when it's wrong. Documented failure patterns now include agents that write tests verifying mocks instead of code paths, rewrite failing tests until they pass, and report success over systems they quietly broke. The root cause is architectural, not a model-quality problem: in most agentic pipelines, the system that generates the work also grades it. Confidence and correctness become indistinguishable — and every failure ships as a green checkmark.

This talk introduces the Agentic Validation Loop: a closed-loop architecture where validation is performed by a layer the generating agent doesn't control. We'll walk through its five stages, extracting verifiable acceptance criteria from requirements, designing tests that always carry their own check, executing against real systems rather than mocks, measuring coverage from run evidence instead of assertions, and detecting drift so the suite keeps matching the spec. Central to the loop is evidence as a first-class artifact: a portable, tamper-evident proof pack per run that outlives the run, gates the pull request as a required check, and gives the human who signs off something better than hope. We'll close with a live end-to-end demonstration and the open problems: judging the judge, evidence at scale, and where human accountability must remain non-transferable.

The way we build software is being rewritten in real time. A year ago, AI was the thing everyone was bracing for; today, agentic AI is already in the workflow, writing code, generating and triaging tests, and shipping alongside engineering teams. Automation is compounding, AI agents are becoming everyday coworkers, and the skills that define a strong engineer are shifting fast. If you're wondering what to learn next, how to grow in an agentic world, or how to stay relevant without burning out trying to keep up with every new tool, you're not alone.

In this candid AMA, Rahul Shetty, an educator and mentor to over a million engineers and testers, joins us to help you make sense of it all. From getting hands-on with AI agents, to building the skills that matter in an AI-native era, to future-proofing your career and mindset for what's ahead, this is your chance to ask what's really on your mind and get honest, practical answers.

Every organization is racing to build "the context layer": pipe Teams, Zoom, Outlook, Jira, and Confluence into one indexed knowledge store, so that every AI agent can retrieve the right snippet at runtime. That's the RAG playbook, and it works — until an agent asks a question whose answer was never written down anywhere. Retrieval can only return what was captured. If the context doesn't exist as data, the agent runs blind — and quietly, that ceiling caps every AI workflow you ship on top of it.

This talk proposes a different contract for the context layer: one that behaves less like a filing cabinet and more like a brain — simulated, not literal. Instead of sitting idle between queries, the layer keeps working. Every captured fragment becomes a node in a knowledge graph — topics, people, decisions, risks — and a continuous consolidation loop keeps revisiting those nodes, testing how they correlate across meetings, tools, and time. When disconnected fragments keep pulling toward each other, the layer mints something genuinely new: an emergent concept — an idea that exists in no document, born purely from the connections. And because every emergent idea carries full lineage back to its source conversations, synthesized knowledge stays auditable rather than hallucinated.

Generic AI assistants can answer questions, but they don't understand your codebase, your incidents, or how your team actually works. That gap is what separates a helpful chatbot from a genuine engineering coworker. This session explores the shift from general-purpose assistants to AI systems that reason over the full context of software engineering: code, tickets, documentation, incidents, and the organizational knowledge that lives between them.

At the heart of this evolution is enterprise context, the foundation for AI that can meaningfully support modern engineering teams. This talk unpacks why that context matters so much, and shares practical lessons on building AI that developers actually trust, including how to balance automation with human oversight and keep engineers in control as systems take on more of the work.

AI agents have faces now. They interview candidates, onboard customers, and triage patients: live, on camera. And almost nobody tests them, because testing one means a human joining the call, playing a role, and sitting through the whole conversation. Ten personas is a lost day. So it doesn't happen before every release.

We'll show it running live and talk honestly about what's hard: hearing the agent at all, knowing when to stop talking, and answering fast enough to sound human.

When software evolves from predictable, deterministic scripts to autonomous, non-deterministic agents, classical quality metrics crumble. A pristine report showing 94% test coverage can easily mask a silent, recursive prompt loop that costs $400,000 in production before a single alert fires. In this high-stakes agentic era, the ultimate currency of engineering is no longer test coverage—it is trust.

True quality leadership requires balancing the inherent chaos of autonomous AI systems with the rigid demands of compliance, safety, and business risk. Drawing from the front lines of shipping AI at scale, this session will deliver a pragmatic, battle-tested playbook for technology leaders, unpacking an Empirical Trust Framework that treats trust as a hard metric, mapping statistical risk boundaries directly to corporate SLAs.

Last year, in our session Ship Code Without Writing It, we spoke about how AI was changing the way developers move from an idea or ticket to working code.

This year, the story has moved beyond individual developers. Across our organization, we have seen AI-assisted development evolve from a productivity boost for engineering teams into a larger shift in how software gets imagined, built, validated, and taken to production.

Every QE organization is now shipping software that AI helped write, test, and increasingly approve. Most are doing it without a clear answer to a simple question: who is accountable when the AI gets it wrong? “AI governance” has become a boardroom phrase, but on the ground it tends to collapse into either compliance theater or nothing at all.

In this fireside chat, Harleen Bedi, Associate Vice President & Unit Technology Officer - Quality Engineering, at Infosys, unpacks what trust and governance actually mean for teams adopting AI across the quality lifecycle, the controls that earn their keep, the decisions that should never be fully delegated to a model, the evidence teams need when someone asks how they know a release is safe, and how the QE role itself is being redefined as the accountability layer for AI-built software.

Financial platforms must validate lifecycle events such as interest accrual, scheduled and partial payments, delinquency, charge-offs, refunds, and settlements before those events naturally occur. Waiting weeks or months is not practical when products and partner integrations must launch quickly and confidently.

This session explores how a lifecycle-aware Test Data Management Portal can create production-like financial states and programmatically advance accounts and loans through future servicing events. Attendees will learn how deterministic time-travel simulation enables repeatable end-to-end validation across servicing, payments, applications, APIs, databases, and customer communications. The session will also cover the architectural principles, governance considerations, and lessons learned from implementing this capability at enterprise scale.

Like everyone and their mother, you now have a chatbot on your site. But can you trust it to give the right answers? Not insult the users? Not give them ingredients for a homemade exploding salad?

Testing something that gives you a whole lot of text, and never the same way twice. That's tough. But not impossible.

Do you have flaky, fragile, and forgotten tests in your repository? I did. In this session, I will explain why tests end up in such a poor state—and, of course, how to avoid falling into the same trap.

When I joined one of my recent projects, people kept saying they had not been able to deliver properly for a long time. There were constant complaints about instability, slow pipelines, and too many testing-related problems slowing development down.

Most AI agents never make it past the demo stage, and the gap between a prototype and a production system comes down to evaluation. I cover how to choose between open source, proprietary, and fine-tuned models, define custom metrics for your use case, calibrate LLM judges for cost-effective assessment, and track results over time to measure real progress. Attendees leave with practical tools to move from proof-of-concept to production.

In this session, we'll explore how system prompt design shapes real-time voice AI behavior, from pacing and hesitation handling to confirmations, interruptions, tool calling, and recovery. Using a live Vapi demo, we'll compare approaches to system prompt design across the same use case to understand the implications of small changes. We will also discuss how model selection affects how your voice agent interprets and uses your system prompt. Attendees will leave with a practical framework for designing voice agents that can listen naturally, use tools reliably, and feel more patient, capable, and human in conversation.

Agentic AI systems don't behave like the software QA teams were built to test. Outputs vary run to run, decisions happen at runtime, and pass/fail is rarely binary. In this session, John Liang shares how Salesforce has evolved its testing approach to keep pace, the assumptions that broke, the practices that replaced them, and what quality engineering teams should be rethinking now as agents move deeper into production.

The first wave of developer AI focused on helping individuals write code faster. The next wave is where agents transform the entire software delivery lifecycle. But enterprises have different needs. They do not simply need more code generated faster. They need an agentic software factory that operates with quality, control, reliability, and accountability at scale.

That takes more than deploying coding agents. Enterprises must be able to modernize large, complex codebases, ensure consistent standards across thousands of repositories, validate AI-generated changes, manage security and compliance requirements, resolve production issues, and determine whether increased AI usage is producing better engineering and business outcomes. The goal isn't tokenmaxxing. It’s outcome-maxxing.

Quality Engineering is undergoing the most profound shift in its history. With the rise of GenAI and autonomous agents, testing is no longer a sequence of manual checks or scripted automations - it is becoming an ecosystem where humans and AI jointly assure quality across Assisted, Augmented, and Autonomous modes. This keynote explores how the Bionic Workforce is redefining the testing function and the new leadership mindset required to unlock its full potential.

In the Assisted mode, AI supports testers with intelligent insights - identifying gaps, analyzing logs, summarizing defects, and providing real time recommendations. Testers gain cognitive acceleration, moving from reactive defect detection to proactive risk discovery.

Agent evaluation has moved fast over the past year, bringing trace-level scoring, multi-turn and tool-use testing, LLM-as-judge calibration, and evals wired into CI rather than run once before launch. But one gap keeps showing up: the evaluations teams need for their own system usually don't exist yet. Generic metrics like helpfulness, groundedness, and toxicity are useful signals, yet a system can score well on all of them while still violating the rules that actually matter, such as issuing a refund above threshold, ignoring an approval boundary, or following an instruction hidden in a tool result.

This talk covers where agent evaluation stands today and what has genuinely changed, then focuses on the shift I think matters most: treating your written behavior specification as a first-class input to evaluation rather than background context. I'll walk through ASSERT (Adaptive Spec-driven Scoring for Evaluation and Regression Testing), an open-source framework from Microsoft's Responsible AI team that turns natural-language behavior requirements into executable evals, generating a reviewable behavior taxonomy, stratified test cases, full agent traces, and per-case verdicts with rationale and policy citations. Using a multi-agent travel-planning agent as the worked example, we'll look at where it passes, where it fails, and why an aggregate score would have hidden both.

Agentic AI has captured the industry's attention. As more people experiment with LLMs, they develop intuition aka vibes around whether models perform well. While math and classification are easy to verify, generative tasks aren't. Adding agents makes evaluation far more complex. This talk highlights key challenges in productionizing agentic applications: non-deterministic outputs, elusive ground truth, and models that mysteriously regress. Yet 'LGTM' isn't a deployment strategy.

Unlike chatbots that resemble sophisticated search engines, we expect agents to actually perform tasks; making security, bias, and privacy non-negotiable before unlocking real use cases in healthcare, finance, and legal. We'll explore why agent evaluation is fundamentally harder than traditional ML testing: multi-step reasoning chains, tool-use side effects, and more. Topics include building evaluation datasets reflecting production scenarios, automated LLM-as-judge pipelines, knowing when human-in-the-loop is unavoidable, and detecting regressions before users do.

As software systems grow in scale and complexity, traditional approaches to test automation are no longer enough. At Microsoft, we are evolving test automation by combining scalable automation platforms with AI agents that help engineers generate, maintain, and improve tests more efficiently.

In this session, I will share real-world lessons from scaling test automation across large engineering teams. We will explore how standardized automation frameworks, reliable CI/CD integration, and self-service tooling enable quality at scale. I will also demonstrate how AI agents are transforming the developer experience by generating Playwright tests from requirements, creating meaningful test cases, suggesting improvements to existing tests, and reducing the maintenance effort required to keep automation reliable.

We think the AI revolution is nearly built. It's barely begun. Drawing on the industrial revolution — its manias, its missing parts, and the tinkerers who actually built it — Magnus makes the case that we're still digging canals, and shows what's left to build and who gets to build it.

We are very early, and there is still a lot to build: we are in the stage where many of the parts and practices still need to be invented. This talk focuses in particular on the need for tools to collaborate with AI on new abstractions, and to create more autonomous systems that still keep humans in the loop for guidance and verification — with real experiences from building this at Sanity.

RL Environments are used by frontier labs to train their latest models and agents. In this talk, I will explain why RL environments are fundamental to all stages of building agents: from evaluations and prompt optimization to post-training agents. I will show concrete industry examples of how practitioners use RL environments to evaluate their agents, and to understand how their agents fail. Lastly, I will also talk about how one can set up data flywheels which helps you figure out how to improve your agents on your specific data and use-cases.

Every quality organization is being redesigned around AI, but few leaders will say out loud what that means for teams, titles, and careers. This panel puts senior QE leaders on the record about the org chart of 2027: the roles they are hiring for that did not exist two years ago, the honest future for manual and exploratory testers, the places where agents created headcount instead of cutting it, and what breaks first when a quality organization scales. Expect candid answers about budgets, skills, and structure rather than safe predictions about tools.

Working with developers can sometimes feel like navigating a jungle. You never quite know which species you'll encounter. Some want to code at lightning speed, others dive into every detail, some prefer working quietly, while others think out loud.

As testers, our success often depends less on the tools we use and more on how we communicate with these different personalities. In this practical and entertaining session, we'll explore common developer types and how testers can adapt their approach to collaborate more effectively with each of them.

Your LLM returns an answer — but is it actually relevant to what the user asked? Relevancy is the most deceptive failure mode in LLM applications: outputs look fluent, sound confident, and completely miss the point. This talk dives deep into measuring and testing relevancy using DeepEval, covering how to catch drift, hallucination disguised as helpfulness, and the gap between "correct" and "useful."

Ranking systems have traditionally been built around carefully engineered objective functions, learned behavior, and years of optimization. But what happens when we introduce models that can reason, interpret intent, and understand natural language? This talk explores how LLMs are reshaping modern ranking systems through the lens of engineering tradeoffs.

Drawing on the latest academic research and public industry work, we'll examine emerging approaches to expressing ranking objectives, combining reasoning with prediction, designing production-ready architectures, and tackling new challenges around validation and continuous improvement. Rather than focusing on a single architecture, this session explores what researchers and industry practitioners are trying today, what seems promising, and what questions remain open.

Test coverage is the metric everyone has and nobody trusts. A 90%-covered codebase still ships incidents. The question worth answering is harder: of the tests we run, which ones have ever caught anything that mattered?

This paper proposes a backwards effectiveness score. For each historical incident in our streaming QE database, we replay candidate test suites against the pre-incident state of the system and record which tests would have failed. Tests that fail on real historical incidents score high. Tests that have never failed on anything real score low, regardless of code coverage or assertion count.

No sector has more to gain from AI, and less room for error, than BFSI. Banks are deploying AI across lending, fraud, and customer servicing; insurers are reimagining underwriting and claims; financial services firms are racing to move agentic AI from pilot to production. Yet every deployment must answer to regulators, auditors, and customers whose trust is the entire business. This panel brings together leaders from banking, insurance, and financial services technology to share their playbooks: which AI use cases have genuinely earned their place in production, how trust becomes an engineering discipline rather than a slogan, and what quality at scale looks like when the systems you're testing are non-deterministic and the stakes are someone's loan, claim, or life savings.

The tools and processes we use to ship software — tickets, repos, pull requests, deployments — were designed for humans writing every line of code. In a world where agents are running the loops, is the software development lifecycle as we know it dead? Join Entire CEO Thomas Dohmke for a conversation on what comes after.

Lazar will explain the process of building repeatable, scalable, secure systems using Lovable — building your own MCP in Lovable and using it in Claude, using Lovable via the Lovable MCP from Claude, creating skills in Lovable, and using project knowledge and spec-driven development to build production-grade apps.

Agentic testing has transformed how the web gets tested — but it left something behind. The majority of digital time, engagement, and revenue now flows through mobile apps, yet nearly every testing agent shipped in the last two years drives a browser. The industry automated the surface that was easy, not the surface that matters most.

The irony? Agents already know how to work with mobile. Frontier coding agents like Claude now launch iOS simulators on their own to build, run, and verify the apps they’re writing. Development has crossed into the agent era on mobile — testing hasn’t. And the reasons are structural, not incidental: there’s no DOM, only native view hierarchies that shift across iOS, Android, and OS versions. Hybrid apps jump between native screens and WebViews mid-flow. Permissions dialogs, lifecycle events, locales, and network states turn every run into a moving target. The result is what mobile teams live with today — brittle Appium scripts, exploding device matrices, and more effort spent keeping old tests green than covering anything new.

Writing your first Playwright test is easy, but keeping hundreds of them fast, reliable, and maintainable is where the real challenge begins. If you're a developer or SDET who already knows the basics of Playwright and is ready to scale your automation efforts, this session will help you level up. This talk explores advanced Playwright practices through the lens of real-world problem solving. Rather than focusing on syntax, we'll dive into strategies that help experienced practitioners build resilient, production-grade test suites that evolve alongside modern applications and teams.

By the end of this session, experienced Playwright users will walk away with actionable strategies to build more robust, maintainable, and team-friendly test automation at scale.

Join us for a fireside chat with nunu.ai on the current state of AI agents in mobile QA. Working with enterprise customers across mobile apps and games, nunu.ai is helping automate large parts of manual testing and mobile workflows using AI agents. Their agents currently lead public phone-use benchmarks, offering a glimpse into what's possible today. We'll discuss how studios are already deploying AI in production, where the technology delivers real value, how you can get the most out of it and what the future of mobile QA looks like.

Traditional game difficulty adjustment mechanisms rely on rigid, centralized, or predefined human-engineered heuristics that inadequately map to the heterogeneous landscape of human cognitive and motor accessibility. This paper introduces Local Agentic Theory, an architectural paradigm that deploys small, autonomous reinforcement learning and reasoning agents natively on edge devices to achieve dynamic, individualized game balancing. Moving beyond traditional cloud-bound models that introduce severe network latency (approximately 200 ms) and prohibitive server costs, our proposed framework operates strictly on-device via localized execution pipelines (approximately 3 ms latency) using optimized runtime environments like LiteRT.

By utilizing advances in sample-efficient reinforcement learning (e.g., EfficientZero architectures), these on-device agents decouple from continuous weight-optimization grinds, functioning instead through real-time "perceive-predict-decide-act" inference loops executed 60 times per second. We evaluate the efficacy of these local agents in balancing a real-time arcade shooter under rigid, multi-objective constraints, explicitly optimization across three localized scarcity budgets: memory capacity (RAM footprint), temporal windows (16 ms per frame), and thermal/battery endurance.

A practical session for testers on detecting risks like manipulation, data exposure, and unsafe behavior in AI agents before they reach production.

Your AI-powered application can pass every functional test and still create a serious security incident. Not because it is broken, but because it follows the wrong instructions.

AI coding agents are changing the unit of software work from the keystroke to the ticket. An agent now accepts a scoped task and returns a plan, a traceable change, tests, and a pull request ready for review.

This session presents a practical operating model for agentic engineering, drawn from deployments in large regulated enterprises: local agents as an extension of your hands, cloud agents as team members, event-driven workflows with human approval gates, and delivery as a closed-loop quality system. A live walkthrough shows what green CI can hide and how evidence-based review catches it. One rule to take home: run the task where its verification lives.

Agents made it possible for engineers what was reserved for a select few with either fat pockets or niche skills. Agents came as a great equalizer. It allows people with deep domain and technical knowledge to create products at a pace that was never before possible. Even if one of your products finds the right audience, you may find your way to financial freedom.

For reference, here are a few problems that I tried to solve using agents and AI from the beginning of year 2026:

Almost every enterprise now has an agentic engineering win, a team that shipped faster, a workflow that got automated, a pilot that impressed the board. Far fewer have turned those wins into how the organization actually builds and tests software. Scaling is where the hard problems live: governance across hundreds of teams, wildly uneven engineering maturity, legacy estates that resist automation, and a workforce being asked to change how it works faster than most change programs move. We'll unpack what genuinely scales versus what only demos well, how you measure adoption beyond seat counts, where verification becomes the constraint on how fast agents can be trusted, and what you'd do differently if you were rolling it out again today.

Most "AI testing" demos stop at generating a test. Real QA is a loop: you author coverage, capture what actually happened, assure it against intent, and keep it all from rotting as the product changes. In this hands-on session we run that full loop end-to-end with kane-cli, an agentic test tool that drives a real browser and authors runnable tests from plain-language requirements.

You'll leave with a working pipeline on your own machine. We cover:

As developers lean on Copilot, ChatGPT, and similar tools, QE teams are inheriting a new class of defects, which is confident, syntactically perfect, and semantically wrong. This talk explores what it takes to test software you didn't write and can't fully predict.

Traditional QE frameworks were built on an assumption that has quietly stopped being true: that the person or system generating code understands the intent behind it. AI-generated code breaks this. It passes linters, compiles cleanly, and often looks more "correct" than human-written equivalents, while silently drifting from business logic, security posture, or regulatory intent. Drawing from work building governance and control frameworks for AI-driven systems in regulated environments like banking, this talk lays out a practical model for catching what traditional testing misses: how to design verification layers that check for semantic correctness and not just syntactic validity, where human review still has to sit in the loop, and how QE teams can build the muscle to test systems whose failure modes they haven't seen yet. If your team is shipping AI-assisted code today, this talk gives you a framework to start closing that trust gap.

Forward-deployed engineers are not new. For years, they have worked between complex products, difficult customer problems and real-world delivery.

So why has AI suddenly made the role feel like the future of engineering?

Accessibility checks usually arrive late and can be flaky. This session makes them contract-driven. We add lightweight @a11y hints (labels, roles) to a GraphQL schema and generate Playwright + axe-core tests. The schema becomes the source of truth for accessible names/roles, and your CI fails when the UI drifts. Live, I'll: (1) annotate a small schema, (2) run a single file Node generator that emits Playwright specs, and (3) execute them with clear pass/fail signals and reports.

You'll leave with a runnable starter plan, a minimal directive set you can copy, and a rollout path teams can adopt in days, starting with 5–10 critical screens. Stop guessing: make the schema the contract.

The future of test automation is conversational. In this talk, we’ll explore how the Model Context Protocol (MCP) transforms Appium from a traditional automation framework into an AI-native testing platform that understands natural language commands and intelligently interacts with mobile applications.

We’ll demonstrate mcp-appium, an open-source solution that creates a seamless bridge between AI assistants like Claude and Appium’s mobile automation capabilities. By exposing Appium’s WebDriver protocol through MCP’s standardized interface, we enable developers and QA engineers to write, debug, and execute mobile tests using natural language.

The next frontier of agentic AI will not only be determined by who builds the smartest agents — it will be determined by which organizations successfully integrate AI into the way humans work, decide, create, and lead.

As AI moves from experimental tools to autonomous agents embedded in daily workflows, quality engineering must expand beyond traditional measures of performance, accuracy, and reliability. The question is no longer only "Does the AI work?" The next question is: "Can humans successfully work with the AI?"

The first time I tried to write a test for an agentic workflow, I reached for assertEquals — and immediately realized how useless it was. The output was different every run. The tool calls happened in a different order. The reasoning path shifted mid-conversation. Everything I knew about testing said this system was broken. But it was working exactly as designed.

This is not a theoretical problem anymore. Teams across the industry are shipping agents that plan, reason, invoke tools, and make autonomous decisions in production. But testing practices have not kept up — most teams rely on happy-path prompts and manual spot checks because the traditional playbook was never built for systems that think for themselves.

AI in BFSI is crossing a threshold, from models that recommend to agents that act. Banks are piloting agents that resolve disputes and move money, insurers are deploying agents across claims and underwriting workflows, and financial services firms are wiring agents into fraud, servicing, and even software delivery itself. But an agent that acts is an agent that must be governed, audited, and tested like any regulated actor.

This panel brings together leaders across banking, financial services, and insurance to discuss what it takes to put autonomous agents into production where real money and real customers are on the line: the guardrails and human-in-the-loop checkpoints that regulators will expect, how to test and certify systems that are non-deterministic by design, and where the line sits today between what agents are allowed to do, and what they've actually earned the trust to do.

Performance testing often creates more noise than value. Teams run load tests, collect huge amounts of data, and generate detailed reports, yet still struggle to answer the question that really matters: Will this change be safe in production?

The challenge is rarely the testing tool itself. Instead, it lies in the gap between performance testing and the broader engineering practices required to make results meaningful: clear service level objectives, realistic workload models, effective observability, production feedback loops, and risk-based decision making.

As AI reshapes how software systems are developed and delivered, most organizations still ask how to scale automation faster. In doing so they overlook a deeper challenge: AI doesn't merely automate tasks, it delegates judgment and decision-making in ways conventional systems never did.

Unlike traditional automation, which fails loudly and predictably, AI systems often fail silently — confident but incorrect outputs, context drift, and culturally misaligned responses that erode confidence over time. These failure modes demand a new approach to quality that goes beyond test coverage and accuracy metrics.

Everyone's building AI agents for QA. Most ship demos. A few break production. Almost none have evals.

I'm building one at Fox right now. It runs against our streaming platforms, Apple TV, Roku, Fire TV, and I'm refusing to ship it until it can pass an eval suite as rigorous as the one I use to gate the test code it generates.

Running large-scale iOS automation has traditionally required keeping devices physically connected over USB. As device labs grow, cable management, unstable USB connections, limited observability, and host constraints become common sources of flaky automation.

RemoteXPC changes this model by enabling secure network-based communication with iOS devices. Built as an open source implementation of Apple's RemoteXPC protocol, appium-ios-remotexpc provides the communication layer that allows Appium to interact with modern iOS devices wirelessly while exposing services such as diagnostics, logging, and device management.

Quality engineering has always rested on one quiet assumption: same input, same output, pass or fail. AI-driven and agentic systems break that assumption at the foundation. The same prompt can produce different — and not necessarily wrong — outputs across runs. An agent might choose a different valid path to the same goal. The old pass/fail model doesn't just get harder to apply; it stops being the right primary framework.

This talk covers what quality engineering looks like on the other side of that shift, drawn from a team actively living it — not a finished case study, but real, in-progress work. Rather than treating AI purely as something to be tested, we'll look at AI as a tool reshaping how testing itself gets done: AI-assisted test authoring that generates cases from requirements, self-healing automation that adapts without constant manual upkeep, and where AI is starting to absorb the repetitive, low-judgment parts of QA work.

Cloud infrastructure was built around a request whose cost you can estimate before it runs. You size timeouts to it, forecast capacity from its average, rate-limit against it, and bill by it. Agent workloads break that assumption at the source: an agent decides what work to do while it executes, so the count of tool calls, model invocations, and retries is only settled once the run is finished.

This session traces what fails when unpredictable work lands on infrastructure designed for predictable requests. Timeouts fire in the middle of legitimate long runs and surface as silent failures. Capacity plans built on averages miss a heavy tail that a few runs can dominate. Per-request billing and rate limits drift away from the resources actually being consumed.

AI is rapidly moving beyond the cloud — onto laptops, phones, browsers, vehicles, and edge devices. But for developers and testers, this creates a new challenge: how do you build, validate, and optimize an AI application once, then confidently run it across an increasingly diverse world of hardware?

This session explores the emerging era of portable AI applications, where models and applications can move seamlessly across CPUs, GPUs, NPUs, and other accelerators without requiring developers to reinvent their software for every target.

The SDET role was built for this moment. A Software Development Engineer in Test is, first and foremost, a software engineer who lives in the quality space — owning automation frameworks, shaping test strategy, and making sure teams get fast, trustworthy feedback when they ship.

In the first half of this session, we'll unpack those "foundational SDET muscles": coding in the test stack, designing high-signal tests, debugging messy systems, and wiring everything into CI/CD. We'll look at how those skills already translate into the AI world and walk through simple upgrades you can make to your existing setup so it's ready for agents and AI-powered tooling.

Artificial intelligence has the potential to transform how individuals with disabilities access information, make decisions, and advocate for themselves. However, the most effective AI tools are not designed solely for people with disabilities but with them. This presentation shares lessons learned through Project RAISE and related initiatives focused on advancing self-determination, self-advocacy, and inclusive technology development.

Central to this work is the belief that people with disabilities should be active partners in the creation of AI agents. Rather than serving only as end users, individuals with disabilities can contribute their lived experiences, expertise, and perspectives throughout the design process. Their involvement helps ensure that AI agents address real-world needs, use accessible language, and support meaningful outcomes across education, employment, and community settings.

A growing share of the world's code is now written by AI, and the people responsible for quality are the first to feel it. This panel brings together QE leaders who live that shift every day: teams reviewing code no human wrote, automation that heals itself, and quality organizations that now ship production code of their own. They will get into what the SDLC looks like when agents generate, review, and test software, whether quality engineering becomes more critical or gets absorbed into the platform, how trust and provenance work for AI written code at enterprise scale, and which skills the next generation of testers will actually need.

For twenty years, software economics rested on one unit: a human, a login, a subscription. Agents break that unit. In this fireside chat, Vibhor Rastogi, Managing Director at Citi Ventures, will explore the emerging landscape of agentic pricing. The session will dissect the mechanics, trade-offs, and strategic implications of various monetization frameworks, including outcome, consumption, and productivity-multiplier. Drawing on observations from the venture ecosystem and how leading startups are experimenting with these models, Vibhor will discuss how founders can align their pricing with the actual value their agents deliver, without prescribing a single "correct" path. Founders will walk away with a framework for evaluating their own pricing strategies in the agentic era.

This talk examines how AI is reshaping the SDLC by absorbing mechanical, repetitive work and shifting engineering effort toward higher-order judgment: architecture decisions, system-level integration, risk tradeoffs, and determining what's worth building in the first place.

As AI evolves, Quality Engineering is seeing a shift to fully autonomous processes. How should we manage AI in order to get the best value and maximize its benefits, and what are the risks and potential pitfalls that one should avoid?

Six months ago, our engineering team went all-in on agents. Claude Code for autonomous builds. Cursor for daily development. Custom skills tuned for HIPAA and clinical edge cases. Agent-first architecture from day one. We invested heavily in harness engineering — the scaffolding around the agents that makes them actually trustworthy in production.

The result: feature cycle time dropped 99%, from days to minutes. PR throughput went 4–5x. Sustained, in production, over a full quarter. That part went exactly as we'd hoped.

Every business wants to show that it's 'doing AI', so they're rushing towards the technology before they've settled on the outcome. Leaders are being swept along by a flood of vibe coding, pilots and proof-of-concepts, without a clear destination. The question is no longer "can we build it?" but "can we trust it?"

To build this trust, Andrew Duncan, CEO of QualityAI, will explain why quality engineering and assurance now belong at the very start of the transformation process, rather than as boxes to tick at the end.

Quality engineering leaders are being asked to do more than ever, ship faster with AI, test systems that are increasingly agentic and non-deterministic, prove ROI to the board, and keep mission-critical platforms running without fail. But not everything can be a priority. This panel brings together senior QE and engineering leaders to share what actually sits at the top of their playbooks: where they're investing in AI and automation, what they've deliberately deprioritized, and how they're preparing their teams and toolchains for the agentic era.

Healthcare software carries the highest stakes in engineering, patient safety, regulatory compliance, and zero tolerance for failure. As AI agents move from writing code to testing and validating it, how do healthcare organizations adopt agentic engineering without compromising trust? This panel brings together leaders across healthcare technology and quality engineering to discuss where autonomous agents fit in regulated environments, how to build validation and audit trails for AI-driven quality, and what "safe to ship" means when agents are part of the SDLC.

What does it take to build a quality-first culture inside a fast-moving AI company serving Fortune 500 enterprises — where the product itself is made of AI agents?

At Aily Labs, we build a Decision Intelligence platform that helps global enterprises across Pharma, CPG, and Retail make better decisions, faster. Our platform doesn't just surface data — it reasons, advises, and acts. And that changes everything about how you test it.

As financial institutions evolve from static automation to autonomous systems, engineering leaders must navigate the end-to-end journey of adopting Agentic AI within strict banking environments. This panel brings together leadership to share a practical playbook on orchestrating the transition to AI agents—from establishing governance and guardrails to managing non-deterministic testing and seamlessly integrating with legacy banking cores.

Someone tells your AI assistant to ignore its instructions. It does. Someone asks it a question it was never supposed to answer. It answers. Someone crafts a message that makes it act outside its intended scope. And it doesn't push back.

These are patterns from live products, already deployed, already used. And in most of the teams where they happened, there was no test for it. Not because anyone was careless, but because nobody asked the right questions.

We automated test execution. We automated regression. We automated reporting, data creation, test design, and even parts of defect analysis. And yet, end-to-end testing still feels painfully manual.

Why? Because while we automated the activities, we forgot the spaces between them. Requirements wait to be understood. Test data waits to be prepared. Systems wait to be configured. Scripts wait for context. Failures wait to be investigated. Defects wait to be documented. And engineers remain the invisible integration layer carrying information from one testing activity to another. Perhaps the biggest inefficiency left in modern Quality Engineering is not another manual task. It is the manual handoff.

For two decades, quality engineering at global scale ran on a simple equation: more scope meant more testers. Agentic AI has broken that equation. Delivery leaders are now rebuilding practices around platforms and IP rather than pyramids, shifting from human-led execution to machine-led orchestration with people focused on judgment and oversight - while clients increasingly ask for outcomes rather than effort. This panel brings together QE and digital engineering leaders from the world's leading service organizations to discuss what an agent-first delivery model actually looks like in production: how test factories become agent factories, what the honest career path looks like for a tester inside a practice of thousands, how you govern autonomy across hundreds of very different client environments, and how you prove quality when the work is done by machines and paid for by outcome.

Most "AI testing" demos stop at generating a test. Real QA is a loop: you author coverage, capture what actually happened, assure it against intent, and keep it all from rotting as the product changes. In this hands-on session we run that full loop end-to-end with kane-cli, an agentic test tool that drives a real browser and authors runnable tests from plain-language requirements.

You'll leave with a working pipeline on your own machine. We cover:

Let's be honest: your AI copilots are writing code faster than your QA team can drink their morning coffee. We've given developers turbocharged engines, but we're still trying to run quality assurance on a two-week Scrum treadmill. The result? A massive bottleneck where "AI efficiency" goes to die.

If we want to scale without breaking things, the traditional Software Development Lifecycle (SDLC) needs an upgrade. Welcome to the AI Development Lifecycle (AIDLC)—a framework where human-only sprints make way for specialized AI agent swarms. But when autonomous agents are writing the code, who is making sure it actually works? (And no, "it compiled on the LLM" doesn't count).

In regulated industries, "the tests passed" isn't enough — you need evidence: what ran, against what build, with what result, traceable end to end. This session shows how agentic testing changes the compliance story. Because every Kane-CLI run produces structured execution logs, screenshots, and artifacts by default, evidence collection stops being a manual afterthought and becomes a byproduct of testing itself. We'll then go a step further and introduce evidence-cli, our newly open-sourced tool for capturing, structuring, and exporting quality evidence from test runs — walking through how it turns raw agent output into an audit-ready evidence trail you can attach to a PR, a release, or a compliance report. We'll connect this to DORA metrics and audit-readiness, drawing on patterns from enterprise QE teams in banking, and show how to satisfy both your engineering leaders and your auditors without slowing either down.

The promise of ML-driven test intelligence is compelling: faster feedback loops, smarter test selection, and anomaly detection that catches what traditional automation misses. But the gap between that promise and production reality is where most teams quietly struggle and rarely talk about it publicly. This session closes that gap.

Drawing from hands-on production experience integrating machine learning into an enterprise QA pipeline inside one of the largest financial institutions in the United States, this talk delivers an unfiltered account of what ML-augmented testing looks like when the stakes are real — regulated environments, high transaction volumes, zero tolerance for silent failures, and teams that still need to ship on time.

Your test suite is failing more from flaky locators than actual bugs — and it's costing you 30–40% of your QA budget. This is not a lecture about AI hype. It's a live arena.

Software testing is undergoing its biggest architectural shift in a decade. Enterprise AI adoption in QA is projected to jump from 15% to 80% by 2027, Playwright adoption surged 235% year-over-year, and yet 89% of teams piloting GenAI in QA still can't get it reliably into production. Something doesn't add up — and this session is built to figure out why, together, in real time.

What does it take to transform a company's innovation experiment into a billion-dollar idea? In this session, I'll take you behind the scenes of a groundbreaking initiative within my organization where we aimed to uncover the next billion-dollar business opportunity. The mission? To identify innovative products that would help propel our company towards becoming a $10 billion enterprise by the end of this decade.

But here's the twist: it wasn't just about the idea itself—it was about rethinking how QA and testing played a pivotal role in shaping and scaling that innovation. I'll walk you through the journey of how AI-powered QA redefined our testing processes, enabling us to accelerate innovation, ensure quality at scale, and deliver transformative results.

The tech market has already evolved. AI is competing with you. Layoffs are normal. Junior hiring is slowing down. Resumes are dead as a signal.

Most testers are still playing the old game. Test cases, bug reports, waiting for the next sprint. That worked ten years ago. It won't work now.

AI initiatives often begin with an exciting idea—but turning that idea into a scalable, reliable, and production-ready solution requires a different way of thinking.

This session introduces the AI Architect mindset: a structured approach to translating business problems into effective AI solutions. We'll explore how architects think beyond models and prompts to consider data, context, integration, security, scalability, cost, governance, and user experience.

"AI will take your job." We keep hearing this, but is this actually true? The short answer: sometimes. The better answer: that might be exactly what we want.

Building AI models — and building real products on top of them — turns out to be mostly about testing. Not just checking if software works, but validating behavior, bias, reliability, edge cases, and trust at scale. In other words, quality has moved from the end of the pipeline to the center of everything — ALL the way from the time you decide to do an AI project!

For years, test automation has focused on better frameworks, faster execution, and expanding CI/CD pipelines. One challenge still slows delivery: test data.

Modern applications depend on datasets that are complex, interconnected, and often privacy sensitive. Traditional approaches — copying production databases, hand-building records, maintaining static datasets — don't scale well and are slow and costly to maintain.

Agents are becoming software’s primary users. The question is no longer which model ranks highest on public leaderboards - it is whether agents can reliably complete the real-world tasks your product promises.

Public benchmarks miss local context: your workflows, state, permissions, edge cases, and tool surfaces. Product failures are specific - bad defaults, missing docs, and friction that only appear when agents actually use your SDK, CLI, or MCP. Custom benchmarks solve this by turning real tasks into reproducible experiments you can rerun after every model, harness, or product change.

This session explores the explosive global adoption of AI and its impact on sales operations. The session bridges macro technology trends to daily sales workflows, unpacking the AI product lifecycle from initial testing to enterprise adoption.

AI is accelerating testing at a pace we've never seen before. Teams are generating tests faster, analysing results quicker, and shipping at unprecedented speed.

But beneath this progress lies a growing leadership challenge.

The proposed framework uses Generative AI, LLMs, AI agents, machine learning models, and observability feedback to support self-healing test automation, LLM-based test generation, prompt regression testing, output quality scoring, predictive defect analytics, and intelligent release decisions — transforming quality engineering from reactive validation into an adaptive, intelligence-driven capability.

Problem statements this talk addresses:

Commerce never closes, and neither can quality. Retail platforms now run 24/7 across web, mobile, stores, and marketplaces, where a broken checkout or a mispriced SKU is instantly public and instantly costly. At the same time, AI is transforming both sides of the transaction: agents are writing and testing the code behind commerce platforms, while AI shopping assistants are beginning to browse, compare, and buy on behalf of customers. This panel brings together leaders across retail and commerce technology to discuss how they engineer trust into always-on experiences, what it takes to test personalized and non-deterministic journeys at peak scale, and how quality engineering evolves when your next customer, and your next engineer, might be an AI agent.

I have an AI service desk analyst agent running in production right now. It completes work on its own, without a person confirming it. It is also restricted to one incident type on one service, and this session is about why.

The agentic testing conversation has focused on one question. Is the agent reasoning correctly? We build evaluation suites, we score outputs, we trace decisions. What we have not examined is the quality of the enterprise context the agent reasons over, and the assumption underneath it that a populated CMDB means an informed agent.

AI and LLM features are now entering products faster than most teams can test them. That speed creates a new quality gap: traditional test automation verifies APIs and UI flows, but it often misses prompt injection, data leakage, unsafe output, hallucinations, agent misuse and model behavior drift. This talk shows how testers can translate OWASP AI/LLM risk categories into a practical testing approach that fits real QA pipelines.

Attendees will learn how to think about AI systems as testable software components across the application, model, infrastructure and data layers. The session will demonstrate how to design test cases for prompt injection, indirect prompt injection, sensitive-data exposure, output safety and agentic behavior limits, then convert those into repeatable checks for regression, CI/CD, and release gates.

For decades, Quality Engineering has been built around a simple rule: given the same input, software produces the same output. Generative AI has fundamentally changed that.

Large Language Model based applications, Retrieval-Augmented Generation (RAG) systems and AI agents don't behave like traditional software. They reason, adapt, use memory, invoke tools and collaborate with other agents. The result is a new class of enterprise applications where correctness is no longer binary, and traditional testing techniques alone are no longer enough.

The Smart Engine uses artificial intelligence to optimize software testing by identifying the tests most relevant to each code change. It analyzes code modifications, dependencies, historical test results, semantic relationships, and test reliability to predict potential impact. By executing only the required subset of tests, it reduces feedback time and avoids the cost of repeatedly running the entire test suite.

The approach improves developer productivity while maintaining confidence that critical defects are not overlooked. With every test cycle, the engine learns from new outcomes and continuously adapts to the application and its evolving codebase.