Author End-to-End Tests in Natural Language with KaneAI
Plan, Author, and Evolve E2E Tests for Web and Mobile Apps in Natural Language.
- TestMu AI (Formerly LambdaTest)
- /
- Blog
- /
- 13 Examples of AI in Action and How Each One Is Tested
13 Examples of AI in Action and How Each One Is Tested
AI in action means trained models making real decisions. See 13 examples, from Netflix and AlphaFold to Tesla and John Deere, and how each one is tested.
Last Updated on:
AI in action is easy to miss: Netflix picking the first row on a home page, a sprayer deciding which plant to treat, and a lab choosing which predicted protein shape to trust all run on trained models.
Those models also fail in public: the AI Incident Database recorded 362 documented AI incidents in 2025, up from 233 in 2024, according to the 2026 AI Index Responsible AI chapter from Stanford HAI.
The examples below span software, science, transportation, and agriculture, including TestMu AI's own KaneAI. Each entry names the system, the technique behind it, a verified result, and the method its team uses to check that the model works.
Overview
AI in action means a trained model making real decisions inside a product people use, such as Netflix ranking titles for each member or AlphaFold predicting protein structures. The 13 examples below come from software, science, transportation, and farming, and each one comes with the method its team uses to check the model.
What Are Some Examples of AI in Action?
- Recommendation engines: Netflix ranks rows, titles, and even artwork for each member from viewing history, ratings, and time of day, and tests new algorithms in A/B tests on a random subset of members.
- Protein structure prediction: AlphaFold DB offers over 260 million predicted structures, and each prediction carries a per-residue confidence score, pLDDT, that tells researchers which regions to trust.
- Driver assistance: Tesla's Autosteer is an SAE Level 2 feature, and a December 2023 recall of 2,031,220 vehicles shipped an over-the-air update because its controls might not prevent driver misuse.
- Computer vision in farming: John Deere's See & Spray sprays only the weeds its cameras detect, and a three-year University of Arkansas trial measured herbicide reductions of 43 to 59 percent compared with broadcast spraying.
- AI test authoring: TestMu AI's KaneAI turns plain-English steps, PRDs, or Jira tickets into executable tests, and a person reviews the generated plan before any step runs.
How Are AI Systems Like These Tested?
Teams match the check to the cost of a wrong output: offline replay and A/B tests for recommendations, blind benchmarks and confidence scores for predictions, human review for generated content, pilots and field trials for physical systems, and drift and bias monitoring after release.
What Is Artificial Intelligence (AI)?
Artificial intelligence is software that learns patterns from data and uses them to predict, classify, generate, or act, instead of following only rules a programmer wrote. Machine learning is the part of AI that learns from examples, and deep learning is machine learning built on many-layered neural networks.
For a quick visual walkthrough of this concept, watch the video below from TestMu AI.
Subscribe to the TestMu AI YouTube channel for more tutorials on testing and automation.
- Computer vision - detects and classifies objects in camera images; John Deere's See & Spray uses it to find weeds, and Arugga's pollination robot uses it to find flowers.
- Recommendation models - predict which option a person is likely to choose; Netflix uses them to order rows, titles, and artwork.
- Generative models - create images, text, or code from a prompt; Adobe Firefly, DALL-E 2 at Mattel, GS AI Assistant, and KaneAI all generate output.
- Predictive machine learning - forecasts a value from historical and sensor data, such as paper towel sheet length at Procter & Gamble or the production models Thomson Reuters monitors.
- Deep learning for science - AlphaFold predicts a protein's 3D structure from its amino acid sequence.
- Optimization - searches many possible solutions and scores each one; ALICE generates construction schedules for HS2, and Moderna's algorithms optimize mRNA sequences.
13 Examples of AI in Action at a Glance
Use the last column to find the check that fits an AI feature you are building.
| # | Example | What the AI does | How it is tested |
|---|---|---|---|
| 1 | Netflix | Ranks rows, titles, and artwork for each member | Offline replay, then A/B tests |
| 2 | Adobe Firefly | Fills, extends, and removes image content in Photoshop | Licensed training data; Content Credentials on outputs |
| 3 | Mattel and DALL-E 2 | Generates Hot Wheels concept images | Designers review and re-prompt every image |
| 4 | Thomson Reuters | Runs ML models on a shared SageMaker platform | Drift and bias monitoring after release |
| 5 | Procter & Gamble | Predicts paper towel sheet length from line sensors | Pilots, then continuous line monitoring |
| 6 | Goldman Sachs | Gives employees firewalled LLM chatbots | Year-long pilot with more than 10,000 employees |
| 7 | TestMu AI KaneAI | Writes tests from plain English | Human plan review; reviewed self-heals |
| 8 | AlphaFold | Predicts 3D protein structures | CASP blind assessment; per-residue confidence |
| 9 | Moderna | Designs and optimizes mRNA sequences | Lab experiments on the resulting mRNA |
| 10 | Tesla Autopilot | Assists the driver at SAE Level 2, under supervision | Driver supervision; NHTSA recall 23V838 |
| 11 | Align JV and ALICE | Generates construction schedule options | What-if runs against the existing schedule |
| 12 | John Deere See & Spray | Detects weeds and spot-sprays them | Three-year university field trial |
| 13 | Costa Group and Arugga | Finds and pollinates tomato flowers | Remote trial before a 10-hectare deployment |
AI in Technology, Media, and Business
In software and back offices, a weak recommendation or a drifting model costs money and trust before anyone notices. Most of the checks in this group therefore run on data, through A/B tests, monitoring, and long internal pilots.
1. Netflix: Recommendations and Personalized Artwork
Netflix estimates how likely each member is to enjoy each title, then personalizes the rows on the home page, the titles inside each row, and their order. Its help center article on how recommendations work lists the inputs: viewing history, ratings, members with similar tastes, title details, time of day, preferred languages, device, and how long someone watched.
Personalization reaches the artwork as well. In the Netflix Tech Blog post on artwork personalization, a member who watches many romantic movies may see Good Will Hunting with Matt Damon and Minnie Driver, a comedy fan may see Robin Williams, and contextual bandit algorithms choose the image.
- Technique - recommendation models trained on member behavior, plus contextual bandits that learn which artwork works for which member.
- How it is tested - Netflix scores bandit algorithms offline with a replay method on logged data, and tests new algorithms against the production system in A/B tests on a random subset of members.
2. Adobe: Firefly Models in Photoshop
Photoshop's Generative Fill, Expand, and Remove tools use AI to fill in, extend, or erase parts of an image. Adobe's Generative Fill page says the latest versions run on new Firefly models, and Generative Fill now also offers partner models such as Gemini 3 and FLUX.2 pro.
For its own models, the Adobe Firefly page states that training uses licensed content from Adobe Stock and public domain content where copyright has expired, and that Firefly outputs include Content Credentials indicating AI was used.
- Technique - generative image models built into an editing workflow.
- How it is tested - the check travels with the file: Content Credentials record how an image was created or edited, so a publisher or client can confirm AI involvement before using it.
3. Mattel: DALL-E 2 Concepts for Hot Wheels
When Mattel designers were asked to come up with a new Hot Wheels model car, they sought inspiration from OpenAI's DALL-E 2, as a Microsoft Source feature from October 2022 describes. A designer can type "a scale model of a classic car," erase the top of the generated car, and ask DALL-E 2 to make it a convertible.
The designer can keep tweaking the result, trying it in pink or blue or with the soft top on, so the model widens the set of ideas a designer reviews.
- Technique - text-to-image generation from plain-language prompts.
- How it is tested - human review of every output: the designer decides which images are worth refining, which keeps the model in the inspiration stage of the design process.
4. Thomson Reuters: An AI Platform With Drift and Bias Monitoring
Thomson Reuters built its first enterprise AI platform with AWS around five pillars: a data service, an experimentation workspace, a central model registry, a model deployment service, and model monitoring, as the AWS Machine Learning Blog describes.
According to the same AWS post, the company's enterprise governance team mandates and encourages business teams to monitor model performance over time. The platform tracks data and inference drift with SageMaker Model Monitor and model bias with SageMaker Clarify. Which signals to watch for each kind of AI system is covered in the guide to AI monitoring.
- Technique - machine learning models trained, registered, and deployed on Amazon SageMaker.
- How it is tested - continuous monitoring after release: drift and bias metrics, plus custom metrics that help data scientists decide when to retrain a model.
5. Procter & Gamble: Predictive Models on the Paper Towel Line
In a pilot with Microsoft, P&G learned to better predict finished paper towel sheet lengths, which improves its ability to deliver the right amount of product to the consumer, according to the companies' June 2022 digital manufacturing announcement.
Microsoft's announcement says the platform collects data from sensors on the manufacturing line and applies machine learning and predictive analytics through Azure IoT Hub and IoT Edge, with pilot projects in Egypt, India, Japan, and the United States.
- Technique - predictive machine learning on sensor data at the edge of the production line.
- How it is tested - pilots on real lines before scaling, then continuous monitoring: machine telemetry and high-speed analytics watch production lines for early signs of material-flow problems.
6. Goldman Sachs: GS AI Assistant for Employees
Goldman Sachs rolled out GS AI Assistant firm-wide in June 2025 after reportedly testing it for over a year with more than 10,000 employees, nearly one-quarter of its worldwide workforce, as Fortune reported.
Fortune also reported that the assistant gives employees access to several chatbots, including ChatGPT and Gemini, through large language models firewalled within Goldman, and that its tasks include assisting with coding. OpenAI's GDPval benchmark, whose tasks include finance and insurance work, measures how well such models produce deliverables such as spreadsheets, with occupation experts grading each file against one a professional made.
- Technique - large language models deployed behind the company firewall.
- How it is tested - a long internal pilot at scale before the firm-wide rollout, with the models kept inside Goldman to keep sensitive data from leaving.
7. TestMu AI: KaneAI Writes Tests From Plain English
KaneAI is TestMu AI's GenAI-native testing agent. A tester describes a flow in plain English, or hands over a PRD, a Jira ticket, or a GitHub pull request, and KaneAI turns it into executable test steps with proposed assertions.
Generated tests export to Selenium, Playwright, Cypress, or Appium, and they run on TestMu AI's cloud across 3,000+ browser and OS combinations and 50,000+ real devices.
- Technique - a large language model agent that plans, writes, and maintains end-to-end tests.
- How it is tested - a person reviews and approves the generated test plan before anything executes, and when the UI changes, self-healing re-anchors affected steps and surfaces each heal for review.
The AI in software testing guide covers where generated tests hold up and where they fail across the testing lifecycle.
Note: Describe a flow in plain English, review the steps KaneAI generates, and run them on real browsers and devices. Start free with TestMu AI.
AI in Science and Healthcare
Science examples are judged against the physical world: a predicted protein shape or an mRNA design is only useful once an experiment agrees with it. Both examples here pair speed with a way to check the output against laboratory evidence.
8. Google DeepMind and EMBL-EBI: AlphaFold Protein Structures
AlphaFold is an AI system from Google DeepMind that predicts a protein's 3D structure from its amino acid sequence, and the AlphaFold Protein Structure Database, built with EMBL's European Bioinformatics Institute, provides open access to over 260 million predictions.
Its accuracy was established in a blind test. At CASP14 in 2020, participants predicted structures whose experimental results were not yet available, and Google DeepMind reported a median score of 92.4 GDT across all targets, where a score of around 90 is informally considered competitive with experimental methods.
- Technique - a deep learning system for protein structure prediction.
- How it is tested - blind assessment at CASP, plus a confidence score on every prediction: pLDDT, which EMBL-EBI's pLDDT guide describes as a per-residue measure of local confidence scaled from 0 to 100.
9. Moderna: Algorithms Inside the mRNA Design Pipeline
Moderna's chief data and artificial intelligence officer, Dave Johnson, described the company's approach on MIT Sloan Management Review's Me, Myself, and AI podcast in July 2021: "When we think about everything we do at Moderna, we think about this platform capability."
Algorithms translate a target protein's amino acid sequence into an optimal mRNA sequence and then optimize it for production and expression, so scientists start the work by pressing a button. Johnson said robotic automation, digital systems, and AI algorithms took small-scale mRNA production from about 30 manually produced each month to a capacity of about a thousand.
- Technique - sequence design and optimization algorithms running inside an automated research pipeline.
- How it is tested - in the lab: the pipeline exists to produce mRNA "for the scientist to run tests in," and Johnson called an algorithm that predicts efficacy in humans from a molecule's structure "completely unrealistic."
AI in Transportation and Infrastructure
In transportation and construction, a wrong prediction has physical and financial consequences. These systems are checked before and after deployment, in simulation, by the people supervising them, and by regulators.
10. Tesla: Autopilot and Autosteer
Tesla describes Basic Autopilot as a package of SAE Level 2 advanced driver-assistance features in its Part 573 safety recall report to NHTSA, and the driver stays responsible for the vehicle while Autosteer is engaged.
That December 2023 recall covered 2,031,220 vehicles because Autosteer's controls "may not be sufficient to prevent driver misuse," according to NHTSA recall record 23V838, which also warns of an increased crash risk when a driver is unprepared to intervene.
The fix was an over-the-air software update. Tesla's report says it makes visual alerts more prominent, simplifies engaging and disengaging Autosteer, adds checks, and can suspend drivers from Autosteer use if they repeatedly fail to demonstrate continuous and sustained driving responsibility.
- Technique - SAE Level 2 driver assistance, where the software assists and the driver supervises.
- How it is tested - in public use as well as before release: the recall record cites NHTSA investigation EA22002, and the remedy changed how the software monitors driver attention.
11. Align JV and ALICE Technologies: HS2 Construction Scheduling
Align JV, a joint venture of Bouygues Travaux Publics, Sir Robert McAlpine, and VolkerFitzpatrick, is delivering the C1 package of HS2, the UK's high-speed railway, which includes a 3.37 km viaduct. Because HS2 mandated P6 scheduling software, the construction schedule had already been built over the previous three years, so Align used the ALICE construction optioneering platform to test its schedule assumptions, as the ALICE case study on Align JV explains.
According to the case study, the team created dozens of schedule options for the Colne Valley viaduct in ten minutes, ran what-if analyses, and identified more than four months of savings on the finishes work. Align JV and ALICE won Industry Innovation Champion at the British Construction Industry Awards in October 2022 for that work.
- Technique - optimization software that generates and compares construction sequences.
- How it is tested - scenario testing before anyone breaks ground: each option is a what-if run against the existing plan, and the savings went into the schedule to reduce the risk of delays.
AI in Agriculture
Farm AI is measured in the field, where results depend on weather, crop stage, and how operators configure the machine. Both examples here went through field trials before or during commercial use.
12. John Deere: See & Spray Weed Detection
John Deere's See & Spray uses digital cameras on a sprayer boom and machine learning to identify weeds in row crops and spray herbicide only where it finds them. The original concept came from Blue River Technology, a California company that John Deere acquired, as the University of Arkansas System Division of Agriculture explains in its trial write-up.
In a three-year field trial in Arkansas soybeans that started in 2022, researchers at the University of Arkansas System Division of Agriculture recorded herbicide reductions of 43 to 59 percent compared with broadcast spraying. At a low sensitivity setting, though, the Palmer amaranth pigweed population grew by 280 percent each year, because weeds the system missed went on to produce seed.
- Technique - computer vision and machine learning on real-time camera images.
- How it is tested - independent field trials that count missed weeds as well as herbicide saved; the lead researcher concluded that the high-sensitivity setting is best in most scenarios.
13. Costa Group and Arugga: Robotic Tomato Pollination
Costa Group, Australia's leading grower of hydroponic glasshouse tomatoes, agreed in 2021 to deploy Arugga's pollination robots across 10 hectares of its glasshouse crop in Guyra, New South Wales, where peak production means more than one million plants, as CTech reported.
Arugga's Polly+ robot uses cameras and AI-based computer vision to recognize flowers ready for pollination, then applies calibrated air pulses to each selected flower, according to Arugga's technology page. The company reports yields as good as and up to 20% higher than manual pollination in commercial greenhouses.
- Technique - computer vision that classifies individual flowers, paired with a robotic air-pressure mechanism.
- How it is tested - a trial before scale: Arugga's CEO told CTech the company completed a trial remotely from Israel despite Covid-related barriers before the Costa deployment.
How AI Agents Change These Examples
Most of the examples above make one prediction or generate one output per request. AI agents wrap a model in a loop: the agent holds a goal, calls tools, reads the results, and decides the next step until the task is done.
The Model Context Protocol (MCP) is an open-source standard for connecting AI applications to external systems such as local files, databases, and search engines, and its site lists Claude, ChatGPT, Visual Studio Code, and Cursor among the clients that support it.
- Tool calls - the agent picks a tool, reads what it returns, and chooses the next step, so one request becomes a multi-step task.
- Computer use - browser-driving agents click, type, and read rendered pages, so a workflow can be automated without a dedicated API.
- Non-deterministic runs - the same request can take different paths on different runs, so a test checks the final outcome instead of an exact sequence of steps.
- Approval gates - an agent that writes to a production system needs an explicit confirmation step, because a wrong tool call changes real records.
Support, coding, and research agents already run in production, as the real-world AI agent examples roundup shows. Testing an agent that acts means checking what each run changed. The agent's own report is not that check: it can claim to have verified its work when it did not, a failure known as deceptive completion.
TestMu AI's Agent Assurance, a self-serve product that runs from the terminal as rook, derives test scenarios from an agent's code or spec and runs the real agent before release. It grades each criterion against the files, records, and tool calls the run produced, and reports separately what it could not verify.
How Teams Test AI Systems Like These
The right check depends on what a wrong output costs. A weak recommendation costs a click, while a missed weed or a misjudged lane has physical consequences.
| Method | What it catches | Examples above |
|---|---|---|
| Offline replay and A/B tests | Changes that look better on logged data but lose with real users | Netflix |
| Blind benchmarks | Models that only work on cases they have already seen | AlphaFold at CASP14 |
| Confidence thresholds | Individual predictions that should not be trusted | AlphaFold pLDDT |
| Human review gates | Generated output that is wrong, off-brand, or unsafe | Mattel, KaneAI |
| Monitoring after release | Drift and bias as real-world data changes | Thomson Reuters, Procter & Gamble |
| Pilots and field trials | Gaps between controlled tests and real conditions | Goldman Sachs, John Deere, Costa Group |
| Scenario simulation | Plans that break under different assumptions | Align JV and ALICE |
| Regulatory review | Misuse and safety gaps found in public use | Tesla |
Confidence thresholds are the easiest of these to automate. I ran the check below on TestMu AI's cloud grid: it reads the mean pLDDT that AlphaFold publishes for human hemoglobin alpha (P69905) and for the tau protein (P10636), and flags any prediction below 70, the level above which EMBL-EBI's guide says the backbone is usually predicted correctly.
import { chromium } from 'playwright-core';
const caps = {
browserName: 'Chrome',
browserVersion: 'latest',
'LT:Options': {
platform: 'Windows 11',
build: 'AI in Action - AlphaFold confidence check',
name: 'Gate structures on mean pLDDT',
user: process.env.LT_USERNAME,
accessKey: process.env.LT_ACCESS_KEY,
},
};
const browser = await chromium.connect(`wss://cdp.lambdatest.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(caps))}`);
const page = await browser.newPage();
for (const id of ['P69905', 'P10636']) {
await page.goto(`https://alphafold.ebi.ac.uk/api/prediction/${id}`);
const [entry] = JSON.parse(await page.locator('pre').innerText());
const verdict = entry.globalMetricValue >= 70 ? 'PASS' : 'FLAG: low confidence';
console.log(`${id} ${entry.gene} mean pLDDT ${entry.globalMetricValue} very low ${entry.fractionPlddtVeryLow} -> ${verdict}`);
}
await browser.close();The run, build 107764899 on Chrome 154 and Windows 11, printed:
P69905 HBA1 mean pLDDT 98.06 very low 0 -> PASS
P10636 MAPT mean pLDDT 49.22 very low 0.656 -> FLAG: low confidenceHemoglobin alpha passes with a mean pLDDT of 98.06. Tau is flagged at 49.22, with about two-thirds of its residues in the very low band, which EMBL-EBI's guide attributes either to intrinsically disordered regions or to too little information for a confident prediction.
For a conversational assistant such as GS AI Assistant, the replies are what need testing. TestMu AI's Agent Testing generates scenarios from uploaded documents, scores chat and voice agents on 9 quality metrics including hallucination, bias, and context awareness, and returns a Green, Yellow, or Red production-readiness verdict. A public exam score for the underlying model, such as MMLU, describes the model and not the replies of an assistant built on it.
Test types, tools, and step-by-step plans for AI features are covered in the guide to testing AI applications.
Conclusion
Pick the example closest to the AI feature your team is shipping and copy its check, whether that is a confidence threshold, an A/B test, or a human review gate. To try the review-first approach on your own app, follow the KaneAI getting started guide and author your first test in plain English.
Testing an AI agent? TestMu AI's Agent Testing validates chat, voice and IVR agents before your customers talk to them.
Author
Chaitanya Sharma
Blogs: 23
Chaitanya Sharma is an AI Product Manager at TestMu AI, where he builds agentic AI capabilities focused on computer vision and multi-modality, moving testing beyond static script execution toward autonomous, agent-driven workflows. Before TestMu AI he shipped 135+ features at Sprinklr for a no-code community and website builder used by Fortune 500 enterprises including Dell, Samsung, and Polestar. At Policybazaar he led the zero-to-one launch of a digital lending and insurance marketplace embedded in Bahrain's dominant payments app, building a risk-intelligence engine that compressed loan-approval times by 80%. He explored machine learning and NLP through research at the University of Cambridge, and holds a B.Tech from Delhi Technological University.
Reviewer
Samyak Goyal
Reviewer
Samyak Goyal is a Senior Member of Technical Staff at TestMu AI engineering Kane CLI, the command-line tool that runs browser automation from the terminal, where a flow described in natural language executes in a real Chrome browser and returns pass or fail with shareable proof. He is a backend engineer with 4+ years of experience, previously an SDE at Innovaccer, where he built APIs, introduced Kafka, and cut deployment from weeks to hours. Samyak also builds multi-agent systems, skill-orchestration frameworks, and a personal copilot that indexes 200+ microservice repositories.
AI in Action FAQs
Did you find this page helpful?
More Related Blogs
TestMu AI forEnterprise
Get access to solutions built on Enterprise
grade security, privacy, & compliance
- Advanced access controls
- Advanced data retention rules
- Advanced Local Testing
- Premium Support options
- Early access to beta features
- Private Slack Channel
- Unlimited Manual Accessibility DevTools Tests