Inbound vs Outbound Phone Agent Testing

Why Does Call Direction Change the Test Plan?

Call direction sets what the agent can assume about the person on the line. Inbound arrives with a reason already attached, and outbound arrives at someone who was doing something else.

The FCC has confirmed that the Telephone Consumer Protection Act’s restrictions on the use of artificial or prerecorded voice encompass current AI technologies that resemble human voices. An AI voice on an outbound call therefore falls under consent rules that already existed, and the Commission has further AI-specific disclosure rules out for comment on top of them.

• Inbound opening - the caller supplies the reason for the conversation, so the opening seconds are already on topic and assertions can begin at whether the agent captured that reason correctly.
• Outbound opening - the person who answered did not ask for this, so the opening seconds have to do work the inbound caller did for free, before any task logic runs.
• Consent as test data - outbound suites should treat the consent state of the record as a scenario input, so a passing run never assumes a permission the production list may not hold.
• Replayable openings - capture the opening verbatim in both directions, since it is the only part of the call every recipient hears.

The TCPA carries a private right of action to recover actual monetary loss or $500 per violation, whichever is greater, and a court may treble that to as much as $1,500 per violation where the violation was willful or knowing. The statute sets that amount per violation, which is the part worth carrying into a release decision.

• Blast radius - an inbound defect meets one caller at a time, while an outbound defect is a script fault already replicated across the list before anyone opens a transcript.
• No recall - you can pull an inbound agent and route callers to a queue, while calls already placed have no equivalent stop, which makes a staged slice of the list a release requirement.
• Per-violation arithmetic - the statute counts each violation separately, so a mild script error repeated down a list can outweigh a sharp inbound bug that a handful of callers hit.
• Review order - outbound earns a human listening pass on real opening audio before the list runs, and inbound earns continuous watching after release.

The success measures an inbound program reports on all presume a caller who arrived carrying a problem, so outbound needs pass conditions you define.

• Right-party contact - decide what counts as reaching the intended person, and give voicemail, a household member and a wrong number separate scenario outcomes.
• Disclosure completion - assert that whatever your script commits to saying up front actually reached the recipient, including on calls where the person talked over the opening.
• Stop requests - score whether the agent recognizes a request to end contact on the first attempt, in the wording real recipients use.
• Written as criteria - these are pass conditions you define and state in the scenario, so the outbound run reports against the job the campaign exists to do.

What Does an Inbound Suite Have to Prove?

Every inbound assertion eventually points at a destination. The wording along the way gets scored too, and a fluent, courteous call that finished on the wrong queue still failed. Name the destination inside the scenario before the run, as plainly as you write the caller’s opening line.

• Destination written in advance - name the queue, the skill or the self-service outcome the scenario should end on, so the grader has something fixed to compare the branch against.
• Same request, different caller - hold that destination constant and vary the voice and the background conditions underneath it, so a pass shows the routing decision survived the caller and not only the clean take.
• Menu correctness - assert that the key the caller pressed produced the destination the menu offered, read from the digits captured alongside the audio.
• The early key press - a caller who already knows the menu presses while the prompt is still speaking. Assert the digit is accepted mid-prompt and acted on, instead of being collected once the prompt ends or dropped on the floor.
• No key at all - the caller waits the whole menu out in silence. Assert whether the agent repeats the options, sends the call to a default destination or closes it, and make the scenario state which of those three counts as correct.

Real callers rarely deliver one clean intent and stop there. They stack two requests into a sentence, or set off down one path and correct themselves the moment the agent answers. Both shapes turn up constantly on a live line and almost never in a scenario library.

• Two requests in one breath - assert which one the agent acted on and what became of the other, since an agent that serves the first and drops the second can still close the call politely.
• The corrected intent - the caller revises halfway through. Assert the agent leaves the first destination, lands on the second, and carries the detail already collected across with it.
• Read-back on anything alphanumeric - account numbers, reference codes and spelled surnames are where intake breaks first, so assert the agent repeated back what the caller actually said.
• Detail that survives the handoff - when the call goes to a person, assert that what the caller already gave is restated at the transfer, so nobody has to restart the intake from the top.

Intake failures and routing failures want different fixes, so the suite has to separate them on its own. A correct branch reached through a clumsy sentence is a wording defect, and a pleasant call that ended on the wrong queue is a routing defect. The declared destination is what lets a run tell you which one you have.

What Does an Outbound Suite Have to Prove?

Every outbound call ends in a disposition, and the real output of a run is how its calls landed across the set of dispositions you allowed. A suite that returns a verdict per call and stops there has discarded that shape before anyone sees it. Read the run as a population first, then open individual transcripts.

• Build the set from what the fixture can produce - derive the categories from the records and scenarios you hold, rather than from the outcomes the campaign hopes for.
• Write the boundaries down - where a call could honestly be placed in more than a single category, the line between them lives in the grader’s head, and the mix moves with whoever wrote it.
• Attach evidence to each category - record which turn and which position in the speaker-identified transcript assigns a call there, so it survives a scenario rewrite.
• Name the residual - calls the grader could not place need a category of their own, sitting in the report beside the others instead of dropping out of the tally.

An unnamed residual absorbs unrelated things at the same time. A timing bug, an answering party nobody modelled and a gap in the grading rules all settle into it together. Its weight tells you how much is hiding, and its contents carry the finding.

• Open the residual by hand - listen to what landed there after each run and place each call yourself, since that is where a missing category announces itself.
• Treat growth as a design finding - a heavier residual usually means the disposition set no longer describes what the agent does, so the repair is a new category rather than a grading tweak.
• Agree how heavy it may get - decide in advance at what weight the residual makes the rest of the report unreadable, and keep that level beside the disposition set.

A campaign regression can arrive with every individual call still clearing its own checks, which is the case a per-call report has no way to represent. Calls slide from a category you wanted into a category you merely tolerated, the per-call verdicts hold, and the movement shows up only in the shape of the whole run.

• Compare mix against mix - hold this run’s spread of dispositions against the previous run’s, category by category, so a shift that leaves every assertion green still surfaces as a result.
• Hold the target list still - the mix moves when the composition of the list changes, so pin the fixture version you compare against or the shift you spot belongs to your test data.
• Keep the category names stable - splitting or renaming a disposition resets the comparison, so version the set alongside the scenarios and record the run where it changed.
• Read which way the calls travelled - which category gained and which gave up the calls says more than the size of the swing, because the movement points at the path that changed.
• Send the gap back to the fixture - a category sitting heavier in the live campaign than in your runs names a kind of record your target list never carried, and the repair belongs in the fixture.

What Happens in the First Three Seconds?

Most scenario libraries I have read start the clock after the greeting. The transcripts open with a caller intent already on the table, so turn zero rarely gets tested. An inbound agent answers into a request that already exists, and an outbound agent opens cold. The broader workflow for scoring either one is covered in our guide to testing AI calling agents.

• Human or recording - the agent has to commit to a greeting, a pause or a hang-up before it knows whether a person or a voicemail prompt picked up.
• End of hello - the callee may say hello once, or twice with a gap, and the agent has to decide which pause is the handover.
• Identification first - the agent has to place the identification your script opens with at the beginning of the call, ahead of any pitch and ahead of any acknowledgment of the hello.
• Collision - what the agent does when the callee starts talking on top of the opening line. This is barge-in at the open, a separate problem from mid-call interruption.

A voicemail greeting and a live person both open with a short utterance followed by silence, and in that opening moment the two signals look the same. A beep, a long scripted sentence or an answer to a question is what separates them, and all of it arrives after the agent already had to act.

On outbound runs the agent dials a platform-owned pool number that the simulator answers, which puts turn zero on your side of the test.

• Script the pickup as well as the conversation - write the answering side as a crisp hello, a slow hello, a silent pickup, or a greeting that sounds recorded, and run all of them against the same agent build.
• Move the pickup around the decision point - the response timing controls (response delay, interruption patterns, speaking pace) shift where the callee stops talking, so the agent faces a different commit moment in each run.
• Watch a real one - passive listening on a live outbound call shows the open exactly as it happened, with no injected turn changing the timing.

Silence at the open runs on separate controls that answer different questions, and each has to be set deliberately.

• Turn-wait - per-turn endpointing that governs how long the agent waits after a short “hello?” before treating that turn as finished. Set it too short and the agent talks over the second hello.
• Silence timeout - a whole-call, per-scenario slider with a 30 to 180 second range that governs when a call carrying nothing at all ends. Keep it near the bottom of the range for an answering side that never speaks, and higher when the scenario needs the agent to sit through a long recorded greeting.

Write what you expect at the open as an explicit pass condition inside the scenario. The checks are whether the identification came first, whether the agent waited out the second hello, and whether it left a message or kept selling to a recording.

What Can Go Legally Wrong on an Outbound Call?

What follows summarizes US federal law for test design. The binding read belongs with your counsel, since your obligations turn on your call type, your consent basis and the states you dial into.

The useful move for a QA lead is to convert each duty into an assertion with named evidence, so a failing outbound call produces a citation and the captured call for the review meeting.

• Identity first - where your agent’s voice brings the call under the artificial or prerecorded voice rules, 47 CFR 64.1200(b)(1) requires the message to state clearly, at the beginning, the identity of the business, individual or other entity responsible for initiating the call. Assert that name lands before any pitch, evidenced by the opening turn of the transcript.
• Callback number - (b)(2) requires the responsible party’s telephone number during or after the message. Assert that it is spoken, resolves to that party’s own line, and falls outside the 900 range, checked against your dialer’s configured callback number, since test calls originate from pool numbers.
• Two-second opt-out gap - where the message is made under certain exemptions, or where it includes or introduces an advertisement or constitutes telemarketing and is delivered to a residential line or the lines named in (a)(1)(i) to (iii), (b)(3) requires an automated, voice- or key-press-activated opt-out mechanism within two seconds of the (b)(1) identification. Assert it as a timestamp subtraction on the captured audio.
• Calling window - (c)(1) bars a telephone solicitation to a residential subscriber before 8 a.m. or after 9 p.m., local time at the called party’s location. Fixture data decides this one, so seed targets across zones and read the time zone stored on the target record.
• Revocation, honored within a reasonable time - (a)(10) lets a called party revoke consent by any reasonable method, honored within a reasonable time not to exceed ten business days. Assert that the agent recognizes revocation in the called party’s own words and stops selling, and assert the ten-business-day clock against the suppression system.
• Do-not-call duration - (c)(2) requires national registrations to be honored indefinitely, until the consumer cancels or the administrator removes them; a company-specific request is honored for five years under (d)(6). Assert that the agent captured the company-specific request and read it back, and test five-year retention against the store.
• Wireless targets - 47 U.S.C. 227(b)(1)(A)(iii) reaches any number assigned to a cellular telephone service absent an emergency purpose or prior express consent. A call-behavior harness never sees the consent basis attached to the number your dialer selected, so own that assertion upstream.

Tag each scenario with the condition that triggers the (b)(3) duty, and run the timing check only on the scenarios where it applies.

A scored call proves what the agent said and did while it was connected. It cannot prove the consent record behind that number was valid, or that the do-not-call scrub upstream of the dial ran against a current list.

Those are data and process controls, and a suite can score green on every call while both are broken. Give them an owner and an assertion of their own, outside the call harness.

Did the Agent Reach the Right Person?

Outbound opens with a claim the agent makes before anyone speaks, that this number belongs to a specific person. Every assertion in the call inherits that claim, and the claim can be false.

That claim rides on your target list, which makes the list a test fixture. I build it the way I build any fixture, with records chosen to fail.

• Verification precedes disclosure - a common policy line lets the agent identify itself and ask for the target by name, and holds the reason for the call until that person confirms their identity. Write your version down and assert against it.
• The assertion is about order - a collections or clinical reminder call can end on a correct outcome and still have named the balance or the appointment turns before anyone confirmed who was listening. That early disclosure is the incident, and it survives a passing outcome check.
• The check reads the sequence - your assertion compares where the reason-for-call phrase first appears in the speaker-identified transcript against the identity confirmation. A grader reading only the closing turns cannot see this failure.
• Wrong number, stated plainly - the person says they are not the target, so assert the agent stops the script, does not re-pitch or fish for a better number unless your policy permits it, and closes on a wrong-number disposition. Whether it reaches your CRM is a separate assertion in another system.
• The ambiguous middle - someone who knows the target answers and says “she is not here right now.” Assert the agent discloses nothing further and, before hanging up, does not act as though the contact were completed.
• Machine pickup - treat a shared household voicemail as an unconfirmed third party, so the ordering rule applies. Assert what the agent may leave when nobody has confirmed anything.

The ambiguous middle is the easiest to grade wrong, because the agent sounds courteous throughout. It can still have named the account in turn three, so read position in the transcript rather than tone.

• Wrong-party records - the number was reassigned and someone else answers.
• Stale records - the target moved, left the company, or closed the account.
• Shared-line records - a household or front-desk line where a relative or colleague answers and knows the target.
• Near-miss names - the answering party gives a name close to the target’s but not it, so the agent refuses a near match instead of confirming. That is a matching-policy assertion rather than a transcription one.
• Refusal records - the target confirms identity, then declines to continue.
• Correct records - present, because the suite still has to prove a clean call completes. They are not what the fixture is for.

An outbound suite needs synthetic target data as a first-class artifact, versioned alongside the scenarios that use it. Pulling a slice of production records is faster, and it drops real names and account details into a system that logs every turn. On the Phone Outbound path the agent dials a platform-owned pool number that the simulator answers, so the number reached in a test comes from that pool rather than from your list.

How Much Load Should an Outbound Suite Simulate?

Inbound load testing asks where your line breaks, because the demand arrives whether you sized for it or not. Outbound volume is a number you set in a dialer config, so the test question becomes whether the pace you set is safe to run. The same pacing setting decides how many handoff requests land per minute and how fast consent and window checks have to clear.

• Human fallback supply - how many people are logged in and free to accept a transfer during the exact window the campaign dials, which runs below roster headcount once breaks, training and after-call work are subtracted.
• Simultaneous transfer demand - how many conversations can reach a handoff request in the same minute, a curve that peaks later than launch volume and at a different height.
• Compliance state per record - consent status, suppression lists and your permitted calling window are all checked at dial time, and a faster pace compresses those checks into a tighter loop.
• Retry behaviour - a no-answer that requeues immediately multiplies attempted volume beyond the list size you planned around.

A campaign that outruns its human fallback produces calls nobody can escalate to. The agent announces a transfer, the queue has nobody free, and the caller waits in a state your suite never produced because your suite ran conversations in isolation. Raising dialer capacity increases how many callers reach that state, so the remedy is a lower dial rate or a larger staffed pool.

• Test the announcement and the pending-handoff window - what the agent says as it hands off, and how it behaves while the handoff is still pending, is scripted behaviour you can validate in a scenario, independent of the dialer.
• Set the transfer-window timeouts deliberately - the per-scenario Needs Human Transfer toggle suppresses end-call detection during the announcement and widens the silence timeout for that window, so a slow handoff is not scored as a hangup.
• Define the abandon behaviour yourself - decide what the agent does once the wait expires, apologise and end, offer a callback slot, or drop to voicemail, then assert on that choice, because the suite only checks what you specify.
• Pull pacing evidence from the dialer stack - queue depth, staffed-agent counts and abandon events are reported by your telephony and workforce systems, so read them there.

An outbound suite simulates conversations, and the volume question stays with the dialer. Concurrency in Agent Testing is an inbound control: configurable per org, 1 to 50 concurrent inbound calls, default 5, with queueing beyond that capacity. Dialer pacing sits outside that control, so the safe-pace decision stays in your dialer configuration.

• What a call-behaviour harness settles - whether the agent behaves correctly on any single outbound call, including the handoff path, under noise, interruption and timing variation.
• What it leaves open - caller ID reputation, spam labelling and STIR/SHAKEN attestation are outbound surfaces that sit outside a conversation harness entirely.
• Where the two meet - a change to the conversation design changes average handle time, which is an input your pacing assumes, so re-derive pacing after any script change and rerun the behaviour suite before it ships.

Inbound vs Outbound Phone Agent Testing: Key Differences

Every row below follows from the first one. Once you change who places the call, the harness, the fixtures and the gate all have to move with it.

The row teams get wrong most often is test data. An inbound suite that passes its personas is genuinely covered, while an outbound suite that passes a clean target list has only proved the easiest path through the campaign.

How to Split One Phone Suite Into Two

Split the suite by asset. Anything that describes how a caller sounds, or how a finished call gets judged, carries across both directions untouched. Anything that describes who is on the line and why the call exists has to be built twice.

• The persona and noise matrix - point the second suite at the matrix you already maintain. Direction changes who initiates the call, not how the synthetic caller sounds.
• The evaluation layer - score outbound runs against the same voice quality metrics you already use. Evaluation depth is rarely what separates the two suites, so spend the saved effort on the target-list fixtures.
• Call review habits - a reviewer reads a call back the same way in either direction, so the triage routine and reviewer rota transfer. The surface does change. Inbound live call monitoring gives real-time call duration tracking, speaker-identified transcripts and DTMF detection, while the outbound side has passive listening mode, monitoring a live call without injecting into it.
• CI wiring and the release gate - run both directions through one pipeline and gate on one Green / Yellow / Red production-readiness verdict, with Yellow blocking the release. Separate dashboards let a failing outbound run pass on the inbound result alone.

The second suite picks all of that up through configuration, with no new authoring.

The rebuild concentrates in the assets that encode a direction-specific assumption about who is on the other end. The fixture layer is the part that gets underestimated, because inbound never asked anyone to model the person being called. Budget for it before the scenario work starts.

• Scenario openings - an inbound scenario opens on a caller intent. An outbound scenario opens on a target record and a call reason, because the agent has to account for its own presence on the line first.
• Fixtures - outbound needs a target list seeded with wrong numbers, stale records, ambiguous name matches and recipients sitting in several time zones. Time-zone correctness is a property of the record, so the fixture carries it and the assertion reads it.
• Compliance assertions - the outbound suite carries an assertion class the inbound suite has no equivalent for. These check ordering and timing: what was said first, and how much time sat between two events. The disclosure rules themselves sit outside the fixture inventory.
• Harness direction - Phone Inbound has the platform dial the agent’s real phone number or SIP endpoint, while Phone Outbound has the agent dial a platform-owned pool number that the simulator answers. Outbound therefore needs a dedicated number pool with country code selection, and test calls originate from pool numbers, never your production numbers.
• Assertion tolerances - do not reuse a conversational-quality tolerance on an ordering check. Wording assertions want a band and a human review. Ordering and timing assertions are binary, so configure them that way and let a near-miss fail outright.

Fork the scenario definitions, the fixtures and the assertion set, then wire each half into your voice agent regression testing cycle. Leave the matrix, the evaluation layer and the release gate shared across both. The rebuild touches the fixture layer and little else, so the second suite runs on the same platform you already have.

Which Gates Should Differ by Direction?

Freeze a baseline per direction and diff each run against its own. One shared baseline averages the two cohorts together, and whichever direction contributes fewer runs is the one whose regressions vanish inside it.

• Outbound baseline - has to carry turn-zero audio: whatever the agent says before the other side speaks at all, and how long it waits before asking its first question.
• Inbound baseline - starts where the platform dials the agent’s real number or SIP endpoint, set up as described in the inbound phone agent docs, and the agent answers, so the first frames under test are its greeting, and whatever it does when the simulated caller opens over that greeting, if that is how you wrote the scenario.
• Region stamped into the baseline - US or EU origination is enforced at call initiation with no silent fallback, so a baseline recorded on one region is only comparable to runs on that region.
• Pool numbers on the outbound side - test calls originate from platform-owned pool numbers, so the baseline holds when the production set rotates.

Severity is a property of the individual assertion, and you assign it when you write the check. A compliance assertion resolves to fired or did not fire; a service-quality metric is a distribution. One averaged threshold over both lets a high score elsewhere in the run offset a missed disclosure.

• Zero-tolerance gate - binary checks you define: did the exact opening wording your compliance owner specified play, in the position they specified, on every call in the run. A single miss fails the run no matter what the mean says.
• Tolerance band - quality scores that read as a distribution, gated on an agreed floor and a band width set with the team that owns the metric.
• Per-metric confidence - High, Medium or Low sits next to each score, so a borderline metric marked Low gets re-run before it reaches the average.
• Green / Yellow / Red verdict - the production-readiness call a direction-aware gate needs, because Yellow is explicitly not “almost ready”. Treat it as a result to resolve before that direction ships.
• Transfer-window exception - scenarios with Needs Human Transfer enabled suppress end-call detection during the announcement and widen the silence timeout, so a handoff does not book a dropped call against either gate.

Inbound gets watched per call while the call is open; outbound gets watched per run, mostly after it lands.

• Inbound line - queue behaviour as calls stack past the configured concurrency ceiling, containment before escalation, and whether escalation lands where a human would expect it. Live monitoring gives call duration, speaker-identified transcripts and DTMF detection while the call runs.
• Outbound campaign - watched per completed run: whether the opening script your compliance owner signed off played on every call in the run, and whether the agent recovered when the answering side was slow. Whether real calls get answered at all lives in your dialer and carrier records; caller ID reputation and spam labelling are surfaces a call-behaviour harness does not measure.
• Passive listening - monitor live outbound calls without injecting into the conversation, which keeps a call observable end to end without adding a participant to it.

Production recording analysis is how the outbound side stays watched after launch. Upload recorded calls in batches and they are scored on the same metrics as live tests, so a completed campaign becomes regression input for the next run.

Conclusion

The cheapest version of this work starts with sorting rather than building. Spend an afternoon putting your existing phone scenarios into two piles, the ones that assume a caller arrived and the ones that assume you placed the call.

Everything in the first pile can keep its personas, its audio conditions and its scoring. Everything in the second pile needs a target record, an opening assertion and a threshold that does not average a legal failure against a pleasant conversation.

Start with the opening seconds, because that is where the two directions stop resembling each other and where almost no existing suite asserts anything at all.

TestMu AI’s Agent Testing scores both directions through the same metric framework and returns a Green, Yellow or Red production-readiness verdict rather than an average. The outbound phone agent documentation covers the number pools and timing controls those scenarios run on.