Run Performance Tests at Scale

What Breaks When You Scale a Load Test?

Running one load test from one machine is straightforward. Running enough load to represent real traffic means generating it from several machines at once, and that is where the numbers stop meaning what you think they mean.

The trap below is not specific to any one platform. It applies to any setup that spreads load across more than one generator, and it is the single most common reason a capacity number turns out to be wrong.

The mechanism is simple. A thread count written into a .jmx file is a per-generator instruction, not a global one, so every machine you add runs the whole plan again.

• Without overrides - A plan specifying 250 users, run on 3 machines across 2 regions, generates 250 x 3 x 2 for a real total of 1,500 concurrent users against your endpoint.
• With overrides - A target of 500 users across 2 regions with a 100-user-per-machine cap resolves to 3 machines per region carrying roughly 84, 83, and 83 users each, summing to the 500 you asked for.
• Why it hurts twice - The first failure is a false capacity signal, because the system was never tested at the level you recorded. The second is a false pass, when a plan you believed was heavy got split so thin it exercised nothing.

Verify the real number before trusting any result. Compare the sample count in the results against what your plan should have produced.

Load can originate from six documented regions, and the default is East US in Richmond, Virginia. A test that never touches this setting measures latency from Virginia regardless of where your customers actually are.

• West US 2 - Moses Lake, Washington.
• East US - Richmond, Virginia, and the default when nothing is configured.
• Central India - Pune, Maharashtra.
• Southeast Asia - Singapore.
• Brazil South - Sao Paulo State, Brazil.
• Mexico Central - Queretaro State, Mexico.

The 2000-user figure quoted for JMeter is a ceiling the infrastructure can support under favorable conditions, meaning lightweight requests, sensible timeouts, and load spread across enough machines and regions. Treat it as a planning bound rather than a target.

The rest of this guide is a worked solution on HyperExecute, the TestMu AI orchestration cloud, because it provisions the generators for you and exposes the regional split as a configuration field rather than something you wire up yourself. Both documented routes are covered: uploading a JMeter plan or Gatling simulation through the portal, and driving Gatling or k6 from a config file in CI. For a product overview of both routes, see TestMu AI's cloud performance testing.

The Gatling CLI configuration was executed on HyperExecute CLI v0.2.356. If you want conceptual grounding on test types first, the performance testing guide covers load, stress, spike, and soak testing as categories.

What Do You Need Before You Start?

For the upload path, a TestMu AI account and a test file are the whole list. There is no load-generator fleet to provision, because HyperExecute allocates machines per task and releases them when the job ends.

• For JMeter - A .jmx test plan authored in the Apache JMeter GUI. Standard Thread Group is supported by default, and custom thread groups also work.
• For Gatling - Simulation files in .java format, ideally reading their load values from system properties so the portal fields can drive them.
• For the Gatling and k6 CLI path only - The HyperExecute binary and your credentials exported as environment variables.

# Linux
curl -O https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute
chmod +x hyperexecute

# macOS
curl -O https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute
chmod u+x ./hyperexecute

# Windows (PowerShell)
Invoke-WebRequest -Uri "https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe" -OutFile "hyperexecute.exe"

# Credentials, read automatically by the CLI
export LT_USERNAME="your_username"
export LT_ACCESS_KEY="your_access_key"

Confirm the binary before writing config, because schema rules changed between YAML versions and error messages reference the version you are on.

$ ./hyperexecute --version
HyperExecute version 0.2.356

Running JMeter Load Tests

JMeter is the tool the TestMu AI documentation covers most thoroughly, and its documented route is the Projects portal upload below.

Upload a .jmx Plan and Run It

Author the plan in the Apache JMeter GUI as you normally would, then hand the file to HyperExecute and configure the load in a form.

Step 1: Create the project and upload the plan

• Open the HyperExecute Projects dashboard.
• Click New Project.
• Browse to your JMeter test files and upload them.
• Click Save.

Step 2: Configure the load

• Select the .jmx file you want to run.
• Click Run.
• Fill in the configuration fields described below.
• Click Continue, then click Run Test.

Each field changes what your results actually mean, so it is worth knowing what you are setting rather than accepting defaults.

Field	What it controls	Practical note
Total Users	The number of users you intend to test for	Read the load math section before trusting this figure, because it interacts with machine count
Duration (min)	How long the test runs	Pair it with ramp-up rather than setting it alone
Ramp-up Time (min)	The amount of time it should take to reach the peak test load	A realistic ramp separates a real capacity limit from a cold-start artifact
Total Load Distribution	The regions load is generated from, as a percentage of users per region	Defaults to East US, so untouched tests measure latency from Virginia
Machine count	The number of machines used for parallel test execution	This is the multiplier in the load math trap
Split CSV	Splits input data from a CSV file among different threads or regions	Without it every generator replays identical test data
Java Version	The runtime the plan executes on	Java 11 is the default and Java 25 is available
Job Labels	Tags applied to the execution for identification	Makes performance runs filterable alongside functional builds

Two JMeter user properties can be overridden from the same screen, and both are genuinely useful rather than obscure.

• jmeter.save.saveservice.subresults - Setting it false reduces the results file size, which matters a lot on plans with many embedded resources.
• httpclient.socket.https.cps - Simulates constrained bandwidth, where 0 means unlimited. This is how you model a mobile network rather than a datacentre connection.

Step 3: Read the results

Open the Jobs section, where a completed JMeter run exposes five views. The JMeter on HyperExecute documentation covers each screen in detail.

Summary opens on six tiles: virtual users, average response time, average throughput, error percentage, 90th percentile response time, and average bandwidth. Underneath, the Load and Response Time charts share a time axis, and Additional Details lists start time, end time, and every region the load came from.

Reading the two charts together is the whole skill. When the users line keeps climbing while response time climbs with it, you have found the saturation point rather than a flat capacity wall.

Timeline Report plots performance across the run, which is where a gradual degradation curve separates itself from a single spike.

Request Stats breaks the run down per request, so a single slow endpoint stops hiding inside a healthy-looking average.

Errors lists error codes with counts and percentages. Check this before celebrating a fast run, because a high error rate makes throughput look better than it is.

Logs holds the raw execution output, which is where you go when the other four screens agree that something failed but not why.

Running Gatling Load Tests

Gatling has documented portal and CLI routes, and its portal flow is more opinionated than JMeter's, since it asks you to declare what kind of test you are running before it asks for numbers.

Method 1: Upload a Simulation and Run It

Step 1: Create the project and upload the simulation

• Open the HyperExecute Projects dashboard and click New Project.
• Select Gatling as the performance testing framework.
• Browse and upload your Gatling simulation files in .java format.
• Click Save.

Step 2: Choose a test type

Select the simulation, click Run, and pick one of three test types. Each takes different parameters, and picking the wrong one produces a perfectly valid run that answers a question you did not ask.

Test type	What it answers	Parameters it takes
Capacity Test	How far the application scales before it degrades	Duration (min), Initial Users, Final Users, expressed as user arrival rate per second
Stress Test	Where it crashes and whether it recovers	Duration (min), Total Injected Users
Soak Test	Whether it degrades over extended production-like use	Duration (min), Constant User Arrival Rate

The Test Load Criteria panel opens as step 1 of 2, and each parameter has its own toggle, so a field left switched off is simply not applied. Capacity Test asks for a duration and the initial and final user arrival rates per second.

Stress Test replaces those arrival rates with a single total injected user count, which is the population pushed at the system to find its breaking point.

Soak Test holds a constant user arrival rate for the duration, which is the only one of the three that surfaces a slow memory leak.

The distinction between Capacity and Stress is worth getting right. Capacity ramps between two arrival rates to find the degradation point, while Stress injects a total population to find the breaking point, and a memory leak shows up in Soak rather than in either of the other two.

Step 3: Configure load distribution and limits

• Region - The geographic origin of the load, defaulting to East US.
• % of Traffic - The share of load generated from each selected region.
• Max Users per Engine - The per-engine virtual user cap, which defaults to 2000.
• Global Timeout - The maximum job duration, which defaults to 90 minutes.
• Job Labels - Identification tags for the run.

Click Run Test to start the job.

Step 4: Read the results

A finished Gatling run appears in the Jobs section with five tabs, and the last one is the one most teams actually want.

• Job Summary - Duration, execution time, and status.
• Scenarios - Per-scenario execution validation.
• Logs - Maven build output alongside the Gatling simulation output.
• Artifacts - The Gatling HTML reports as files.
• Report - The consolidated HTML report, downloadable in one click.

One authoring detail decides whether the portal fields do anything at all. Your simulation should read its load values from system properties rather than hardcoding them, so the values you type into the form reach the code.

LoadSimulation.java

// Read load parameters from system properties so the portal fields drive the run.
// Recognized injectType values: constantUsersPerSec (default), constantUsers,
// rampUsersPerSec, stressPeakUsers, soakTest, capacityTest.
String injectType = System.getProperty("injectType", "constantUsersPerSec");
String workloadModel = System.getProperty("workloadModel", "open");

int users = Integer.getInteger("users", 10);
int duration = Integer.getInteger("duration", 30);
int rampDuration = Integer.getInteger("rampDuration", 60);
int usersStart = Integer.getInteger("usersStart", users / 2);
int usersEnd = Integer.getInteger("usersEnd", users);

A simulation with hardcoded injection values still runs, and the portal will still accept your numbers, but the run ignores them. That failure is silent, which makes it worth checking before your first real measurement.

---
version: 0.1
runson: linux

autosplit: true
concurrency: 1
scenarioCommandStatusOnly: true

runtime:
  - language: java
    version: "17"

background:
  - mvn spring-boot:run -Dspring-boot.run.main-class=dev.simonverhoeven.gatlingdemo.GatlingDemoApplication || true

pre:
  - mvn dependency:resolve

vars:
  INJECT_TYPE: constantUsersPerSec
  WORKLOAD_MODEL: open
  USERS: 10
  DURATION: 30
  RAMP_DURATION: 60
  USERS_START: 5
  USERS_END: 10

testDiscovery:
  type: raw
  mode: static
  command: echo "Test"

testRunnerCommand: >
  mvn gatling:test
  -Dgatling.jvmArgs="-DinjectType=${INJECT_TYPE}
  -DworkloadModel=${WORKLOAD_MODEL}
  -Dusers=${USERS}
  -Dduration=${DURATION}
  -DrampDuration=${RAMP_DURATION}
  -DusersStart=${USERS_START}
  -DusersEnd=${USERS_END}"

uploadArtefacts:
  - name: TestReport
    path:
      - target/gatling/**

retryOnFailure: true
maxRetries: 1

jobLabel: [Performance, Gatling]

Which Method Should You Use?

Gatling teams usually end up using both, because the two routes solve different problems. The portal answers a question today and the CLI answers it every night without anyone remembering to ask.

The regional split row is the one that decides architecture. If load has to originate from several geographies, the portal is the documented way to configure that, so a CLI-only setup means giving up geographic distribution.

Let an Agent Write the Config

Authoring the YAML is the slowest part of the CLI path, because discovery and runner commands depend on how your repository is laid out. TestMu AI publishes an open-source HyperExecute skill that hands that job to an AI coding agent.

git clone https://github.com/LambdaTest/agent-skills.git .claude/skills/agent-skills

It ships four helper scripts that are runnable on their own, with or without an agent. doctor.js checks readiness, validate-config.js lints the YAML offline, build-command.js prints the correct CLI invocation, and summarize-artifacts.js triages a downloaded artifact tree.

The readiness check is the one worth running first in a new repository or a failing CI runner, because it names what is missing rather than just failing:

doctor.js output

$ node .claude/skills/agent-skills/hyperexecute-skill/scripts/doctor.js --config hyperexecute.yaml
OK: HyperExecute CLI found at ./hyperexecute
OK: HyperExecute CLI is executable
OK: Config file found at hyperexecute.yaml
OK: LT_USERNAME is set
OK: LT_ACCESS_KEY is set
WARN: No .hyperexecuteignore found. Consider excluding secrets, local artifacts, and unrelated monorepo files.
WARN: No .gitignore found. Ensure local secrets are not committed.
HyperExecute doctor finished without blocking issues.

That last warning is worth acting on. Without a .hyperexecuteignore the CLI packages the whole working directory into the uploaded payload, which on a monorepo means slow uploads and a real risk of shipping local secrets to the grid.

The skill also refuses to trigger a paid cloud job without asking, unless you explicitly opt into an autonomous session. That matters when an agent is holding keys that spend execution minutes.

Running k6 Suites

k6 is the third documented tool and is CLI-only, with no portal upload flow. It installs as a runtime addon with a pinned version rather than through a language runtime.

hyperexecute.yaml

---
version: "0.1"
runson: linux

autosplit: true
concurrency: 2

runtime:
    addons:
      - name: k6
        version: "v0.52.0"

env:
  K6_BROWSER_ENABLED: true
  K6_BROWSER_HEADLESS: false
  HE_CONTEXT_K6_SETUP_DEFAULT_BROWSER_PATH: true

pre:
  - k6 version

testDiscovery:
  type: raw
  mode: dynamic
  command: ls tests/*.js

testRunnerCommand: k6 run $test

scenarioCommandStatusOnly: true

jobLabel: [K6, 'HyperExecute', autosplit]

The three environment variables are only needed for browser-mode runs. A protocol-level k6 script does not need them, and leaving them set on a non-browser run is harmless but misleading to the next reader.

If you have not settled on a tool yet, load testing tools compares the options, and the k6 testing tutorial covers writing the scripts themselves.

Troubleshooting

Every row below is a failure hit while testing HyperExecute for this guide, or one the documentation calls out explicitly. They are listed by the symptom your terminal or dashboard actually shows.

One capacity constraint is easy to hit on an aggressive schedule. Each organization has a maximum allowable daily test duration calculated from its provisioned parallel sessions, documented as 6 hours of runtime per parallel session per day.

Long sessions carry their own warning, since the documentation notes that runs exceeding 120 minutes can cause memory and CPU spikes. That is a direct argument for splitting a soak test across machines rather than running one very long task.

Conclusion

Start in the portal. Upload one existing .jmx plan or .java simulation, set a modest user count and a realistic ramp, and run it once to get a baseline you trust before automating anything.

Then reconcile the sample count in the results against what your plan should have generated. Getting that number right is what separates a capacity measurement from a guess, and it is the step teams most often skip.

Once one run is trustworthy, move your Gatling or k6 runs into a pipeline with hyperexecute.yaml and keep every artifact for trend comparison. The getting started with HyperExecute documentation covers account setup and first-job specifics.