For AI agents and LLMs: a machine-readable index is available at llms.txt. A plain-Markdown version of any documentation page is available by appending .md to its URL.
Skip to main content

Inbound Phone Agent Performance Testing With TestMu AI

Performance testing on the TestMu AI Agent Testing Platform measures how an AI phone agent behaves when it handles many calls at the same time. You run a performance test on a phone suite: TestMu AI uses the suite's scenarios to place real phone calls to your agent, holds a target number of concurrent calls for a set period, and reports on load delivery, reliability, and responsiveness. The run page shows the test's current phase live. Because the test places calls to your agent, it applies only to inbound agents, which answer calls.

When to Use Performance Testing​


Choose the test that matches what you want to check.

  • Use a performance test to measure how your agent behaves under concurrent load: whether the requested number of concurrent calls is sustained, whether calls connect and conversations end as expected, and how quickly the agent responds as load changes.
  • Use a standard inbound phone agent test on the same suite to confirm that your agent behaves correctly on an individual call. A performance test does not evaluate conversation content. See Performance Testing Scope and Exclusions.

Performance Testing Prerequisites​


  • A phone suite that contains at least one scenario. To set up scenarios and test suites, see Test an Inbound Phone Agent.
  • Enough available parallels to cover the target load. A parallel is the capacity for one concurrent call. See Parallels.

Configure a Performance Test​


Set two parameters for the test: how many concurrent calls to hold, and for how long.

ParameterDescriptionAccepted range
TargetNumber of concurrent calls to maintain during the hold1 to the number of available parallels
Hold durationLength of the steady-state measurement period1 to 120 minutes. The default is 5.

TestMu AI places every call over live telephony to your agent, using the scenarios, voices, background noise profiles, and personas configured in the suite. It distributes calls evenly across every scenario and configuration combination (voice, noise profile, and persona) in the suite, so the test exercises each scenario.

What Happens During a Performance Test​


Each test moves through four phases: ramp up, hold, drain, and finalizing. The run page shows the current phase live.

PhaseWhat happens
1. Ramp upTestMu AI initiates calls at a rate of 5 per second until it reaches the target. If the test does not reach the target within the expected ramp time plus a 60-second allowance, it proceeds to the hold and records that the ramp timed out.
2. HoldTestMu AI maintains the target load for the configured Hold duration. When a call ends, a replacement call starts immediately, so concurrency stays at the target. This is the only scored phase.
3. DrainTestMu AI initiates no new calls. Calls in progress get up to 60 seconds to complete. TestMu AI ends any call still active after this window and classifies it as Ended by test.
4. FinalizingTestMu AI waits for the final call records from the voice platform (duration, end reason, and cost), then compiles the results.
Measurement window

TestMu AI calculates load, reliability, and latency metrics on the hold phase only, so start-up and wind-down activity does not distort the measurements. The ramp up and drain phases introduce and remove load in a controlled way, and TestMu AI excludes them from scoring.

Performance Test Safeguards​


Performance tests run with the following safeguards.

SafeguardBehavior
Parallel limitA test never exceeds your organization's parallel limit. If not enough parallels are available at start, the test does not launch, and TestMu AI displays the available count.
Platform capacityIf the voice platform reports that it is at capacity, TestMu AI pauses new calls for 5 seconds before retrying. It records each occurrence as a capacity rejection.
Repeated start failuresIf 10 consecutive calls fail to start for reasons other than capacity, TestMu AI stops the test and marks it as failed.
AbortYou can abort a test at any time. TestMu AI stops placing new calls immediately, ends all active calls, including calls still connecting, and compiles results from the data collected.

View Performance Test Results​


A performance test reports on three areas.

AreaQuestion it answers
Load deliveryWas the requested level of concurrent calls sustained for the full test?
ReliabilityDid calls connect, and did conversations end as expected?
ResponsivenessHow quickly did the agent respond to the caller, and how did that change under load?

The results include summary metrics, an outcome for every call, a per-second load timeline, agent response latency percentiles, and a breakdown by concurrency level with automated findings. Summary metrics cover the hold phase only, while call outcomes and the load timeline cover the full test.

To share a run with a stakeholder, send its report through a link that needs no login. The shared view shows aggregate metrics only, with no recordings, transcripts, or call IDs. Turning the feature off for your organization revokes every link it has issued.

Summary Metrics​

The summary reports four metrics for the hold phase.

MetricWhat it shows
Load achievedPercentage of the hold during which concurrent calls were at or above 90% of the target. Reported with the average number of concurrent calls, the total seconds below target, and the number of capacity rejections.
Connection reliabilityConnected calls as a percentage of all calls that reached a final connection result (connected or not connected).
Response latency (hold)P50 and P95 agent response time across all responses recorded during the hold. Reported with the sample count, the call count, and the number of connected calls that received no agent response.
Conversation reliabilityCalls that ended normally as a percentage of all connected calls that have completed.

Call Outcomes​

TestMu AI assigns each call to exactly one outcome, so totals always reconcile.

OutcomeDefinition
SubmittedAll calls the test attempted to place
Dispatch failedThe call could not be initiated
Not connectedThe call was placed but did not connect, for example no answer or busy
ConnectedThe call reached the agent
Ended normallyA connected call ended by the caller, ended by the agent, or transferred by the agent
Ended abnormallyA connected call that ended for any other reason, such as a silence timeout or agent error. TestMu AI displays the reason.
Ended by testA connected call still active when the drain window closed. TestMu AI ends the call, counts it as connected, and excludes it from normal and abnormal endings.
Still activeCalls ringing or connected while the test is running

Load Timeline​

The load timeline is a per-second view of the full test, with the ramp up, hold, and drain phases highlighted. It plots the following.

SeriesWhat it shows
In flightAll active calls, ringing or connected
ConnectedCalls answered by the agent
RingingCalls placed but not yet answered
TargetThe configured concurrency level

Agent Response Latency​

Agent response latency is the time between the end of the simulated caller's speech and the start of the agent's reply. TestMu AI measures it on every conversational turn.

PercentileInterpretation
P50Median response time. Represents the typical caller experience.
P9090% of responses were at or below this value.
P9595% of responses were at or below this value. The primary indicator used for degradation analysis.
P9999% of responses were at or below this value. Represents the slowest experiences callers encounter.
tip

A widening gap between P50 and the upper percentiles as load increases indicates that a subset of callers is experiencing materially longer delays.

The Latency tab shows the total number of calls and the average, highest, and lowest reply latency in colour bands, which you set for each run. Its per-call table updates live while the test runs. Open any call to see each turn: what the caller said, what the agent replied, and the reply latency.

If the results show latency as not supported, see Performance Testing Scope and Exclusions.

Concurrency Analysis​

TestMu AI breaks the results down by concurrency level to identify the point at which performance degrades. It groups calls by the number of concurrent calls active when each call started. The groups are 1, 2 to 5, 6 to 10, 11 to 20, 21 to 50, 51 to 100, 101 to 200, and 201 to 500.

Each group reports the following.

MetricDescription
CallsNumber of calls in the group
Success rate and drop rateShare of completed calls that ended normally (success rate) or dropped (drop rate)
Time to connectAverage time from call initiation to connection
First responseAverage time to the agent's first reply
Response latencyP50, P90, P95, and P99 agent response time
Call durationAverage length of a call
Error distributionCall endings grouped by reason, such as drop, silence timeout, agent error, maximum duration reached, or no answer
CostAverage cost per call

How Automated Findings Are Decided​


TestMu AI derives findings from the concurrency analysis using fixed rules. No AI model is involved, and the same data always produces the same findings.

FindingRule
Breaking pointThe lowest concurrency group where the success rate falls below 80%, or where P95 response latency exceeds 2x the P95 of the lowest group
Latency degradationRatio of P95 response latency at peak concurrency to P95 at the lowest group
Drop rate alertThe lowest concurrency group where the drop rate reaches 15% or higher
Safe operating capacityThe highest concurrency group below the breaking point, or confirmation that no degradation was observed
Confidence thresholds

A concurrency group contributes to findings only when it contains at least 3 calls and at least 10 response samples. TestMu AI still displays groups below these thresholds and marks them as low confidence, so that isolated events are not reported as capacity issues.

Performance Testing Scope and Exclusions​


Performance tests focus on load behavior. The following are outside their scope.

Not includedDetail
Transcript analysisAI does not evaluate performance calls. A performance test produces no validation criteria, scenario pass or fail results, custom metrics, or conversation quality scores. To evaluate conversation content, run the suite as a standard test.
Ramp up and drain scoringTestMu AI calculates summary metrics on the hold phase only.
Agent infrastructure metricsTestMu AI takes measurements from the caller's perspective. It does not observe server resources, model provider limits, or other systems internal to your agent.
Latency without timed messagesResponse latency requires timestamped messages for both caller and agent from the voice platform. When these are unavailable, TestMu AI reports latency as not supported.
Load above the parallel limitTests do not raise your organization's parallel limit. Additional parallels are required to test higher concurrency.

Parallels​


A parallel is the capacity for one concurrent call. The number of available parallels determines the maximum target for a performance test. Parallels in use by other tests in your organization reduce the number available.

Troubleshoot Performance Tests​


SymptomCauseWhat to do
The test does not launch, and the available parallel count is displayedFewer parallels were available at start than the Target needsSet Target to the available count or lower, or add parallels. Parallels in use by other tests in your organization reduce the number available.
The maximum Target is lower than expectedThe target is limited to available parallels, and other tests running in your organization at the same time use parallelsRun the test when fewer parallels are in use by other tests, or add parallels. See Parallels.
The test stops and is marked as failed10 consecutive calls failed to start for reasons other than capacityReview the call outcomes. TestMu AI counts calls that could not be initiated as Dispatch failed.
The test records that the ramp timed outThe test did not reach the target within the expected ramp time plus the 60-second allowanceThe test still proceeds to the hold. Check Load achieved for the percentage of the hold during which concurrent calls were at or above 90% of the target.
Calls show Ended abnormallyThe calls connected but did not end normally, for example because of a silence timeout or agent errorRead the reason displayed for each call. To see the concurrency levels at which these endings occur, check Error distribution in the concurrency analysis.
A concurrency group is marked low confidenceThe group has fewer than 3 calls or fewer than 10 response samplesIncrease the Hold duration or the Target to collect more data.
The results show no validation criteria, scenario pass or fail results, custom metrics, or conversation quality scoresAI does not evaluate performance callsRun the same suite as a standard test to evaluate conversation content.

Terminal First Testing With Kane CLI

Natural language browser & mobile app tests right from terminal.

×
Schedule Your Personal Demo
Kane CLI terminal

Help and Support

Related Articles