Load Testing SaaS Applications: A Noisy Neighbor Experiment

One enterprise tenant starts a bulk export at 9 a.m. on a Monday. The shared database pool fills up, and twenty small customers on the same cluster watch their dashboards hang while the aggregate latency chart still looks fine.

Load testing SaaS applications means modeling exactly that: many tenants with different traffic shapes on shared infrastructure, measured per tenant instead of as one average. I built a small multi-tenant API and loaded it with k6 to measure the effect, and every number in the results section comes from those runs.

What Is SaaS Load Testing?

SaaS load testing is load testing a multi-tenant application with the tenant as the unit of measurement. The workload, the pass criteria and the results are all split by tenant or plan tier, because tenants share databases, queues, caches and rate limits.

It complements functional SaaS testing, which checks tenant isolation and entitlements for correctness. Load testing asks whether each tenant still gets the latency its plan promises while the others compete for the same resources.

Shared Resources Tenants Compete For

A multi-tenant architecture pools resources across tenants, and each shared pool lets one tenant's traffic add to another tenant's latency:

• Database connections - every tenant's queries draw on the same small, fixed set of server connections, so one tenant's long queries leave fewer for everyone else.
• Rate limits - a per-tenant limit protects the platform only if it rejects the tenant over its limit and not the tenants below it.
• Autoscaling - new pods are requested only on the autoscaler's next check, so a burst lands on the existing pods first.
• Background queues - exports, webhooks and scheduled reports share worker pools, and the queue grows before any API latency changes.
• Caches - a heavy tenant can evict small tenants' cache entries, so their reads fall through to the shared database.

The PostgreSQL connection settings documentation says max_connections determines the maximum number of concurrent connections to the database server, and that the default is typically 100 connections.

RFC 6585 defines 429 Too Many Requests: the user has sent too many requests in a given amount of time, and the response may include a Retry-After header saying how long to wait before making a new request.

The Kubernetes HorizontalPodAutoscaler documentation says the controller runs at an interval set by the --horizontal-pod-autoscaler-sync-period parameter, and the default interval is 15 seconds.

The Noisy Neighbor Experiment

To put numbers on the shared-pool problem, I built a mock multi-tenant API in Node.js and loaded it with k6 v2.3.0 on one Windows 11 Pro machine with an AMD Ryzen 7 7735HS (16 threads) and Node.js v24.14.0. Everything ran on localhost, so network latency is not part of the results.

• Shared pool - 8 simulated database connections shared by every tenant; a request holds one for 20 ms on the items endpoint or 80 ms on the export endpoint.
• Small tenants - 20 tenants sending 40 requests per second in total to the items endpoint.
• Heavy tenant - one enterprise tenant running a bulk export at 100 requests per second, enough to occupy the whole pool on its own.
• Timeout - a request that waits more than 2 seconds for a connection gets a 503.
• Isolation mode - with ISOLATION=cap, each tenant may hold at most 2 connections, its extra requests wait in its own queue of 20, and the server answers 429 once that queue is full.

This is the complete server, server.mjs:

// Multi-tenant mock API: every request borrows one connection from a shared pool.
// ISOLATION=off  -> one global FIFO queue for all tenants (no isolation)
// ISOLATION=cap  -> each tenant may hold at most TENANT_CAP connections; its extra
//                   requests wait in its own queue, and get 429 once that queue is full
import http from "node:http";

const PORT = Number(process.env.PORT || 3101);
const POOL_SIZE = Number(process.env.POOL_SIZE || 8);
const TENANT_CAP = Number(process.env.TENANT_CAP || 2);
const TENANT_QUEUE_MAX = Number(process.env.TENANT_QUEUE_MAX || 20);
const WAIT_TIMEOUT_MS = Number(process.env.WAIT_TIMEOUT_MS || 2000);
const ISOLATION = process.env.ISOLATION || "off";
const QUERY_MS = { items: 20, export: 80 }; // simulated query time per endpoint

const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

// A counting semaphore with a FIFO wait queue and an optional queue limit.
function semaphore(size, queueMax = Infinity) {
  let free = size;
  const waiters = [];
  return {
    acquire(timeoutMs) {
      if (free > 0) { free--; return Promise.resolve(true); }
      if (waiters.length >= queueMax) return Promise.resolve("full");
      return new Promise((resolve) => {
        const w = { resolve, timer: setTimeout(() => {
          waiters.splice(waiters.indexOf(w), 1);
          resolve(false);
        }, timeoutMs) };
        waiters.push(w);
      });
    },
    release() {
      const w = waiters.shift();
      if (w) { clearTimeout(w.timer); w.resolve(true); } else free++;
    },
  };
}

const pool = semaphore(POOL_SIZE);
const tenantGates = new Map();
const gateFor = (tenant) => {
  if (!tenantGates.has(tenant)) tenantGates.set(tenant, semaphore(TENANT_CAP, TENANT_QUEUE_MAX));
  return tenantGates.get(tenant);
};

http.createServer(async (req, res) => {
  const m = req.url.match(/^\/t\/([\w-]+)\/(items|export)$/);
  if (!m) { res.writeHead(404).end(); return; }
  const [, tenant, endpoint] = m;

  const gate = ISOLATION === "cap" ? gateFor(tenant) : null;
  if (gate) {
    const ok = await gate.acquire(WAIT_TIMEOUT_MS);
    if (ok === "full") { res.writeHead(429, { "retry-after": "1" }).end(); return; }
    if (!ok) { res.writeHead(503).end(); return; }
  }
  const got = await pool.acquire(WAIT_TIMEOUT_MS);
  if (!got) { gate?.release(); res.writeHead(503).end(); return; }

  await sleep(QUERY_MS[endpoint]); // the "query" holds the connection
  pool.release();
  gate?.release();
  res.writeHead(200, { "content-type": "application/json" });
  res.end(JSON.stringify({ tenant, endpoint }));
}).listen(PORT, "127.0.0.1", () =>
  console.log(`isolation=${ISOLATION} pool=${POOL_SIZE} cap=${TENANT_CAP} on :${PORT}`));

The k6 script drives each tenant group with the constant-arrival-rate executor, so traffic keeps arriving at the same rate while the server slows down.

The k6 constant-arrival-rate documentation says this executor starts a fixed number of iterations over a specified period of time, and that its iterations start independently of system response.

// k6 tenant-mix test: 20 small tenants at a steady rate, plus one heavy tenant
// running a bulk export. HEAVY_RATE=0 runs the small tenants alone.
import http from "k6/http";

const BASE = __ENV.BASE_URL || "http://127.0.0.1:3101";
const HEAVY_RATE = Number(__ENV.HEAVY_RATE || 0);
const DURATION = __ENV.DURATION || "60s";

const scenarios = {
  small_tenants: {
    executor: "constant-arrival-rate",
    exec: "smallTenant",
    rate: 40, timeUnit: "1s", duration: DURATION,
    preAllocatedVUs: 50, maxVUs: 400,
  },
};
if (HEAVY_RATE > 0) {
  scenarios.heavy_tenant = {
    executor: "constant-arrival-rate",
    exec: "heavyTenant",
    rate: HEAVY_RATE, timeUnit: "1s", duration: DURATION,
    preAllocatedVUs: 100, maxVUs: 600,
  };
}

export const options = {
  scenarios,
  summaryTrendStats: ["avg", "med", "p(95)", "p(99)", "max"],
  thresholds: {
    // the SLO we promise every small tenant, whoever else is on the box
    "http_req_duration{scenario:small_tenants}": ["p(95)<150"],
    "http_req_failed{scenario:small_tenants}": ["rate<0.01"],
  },
};

export function smallTenant() {
  const id = Math.floor(Math.random() * 20) + 1;
  http.get(`${BASE}/t/small-${id}/items`);
}

export function heavyTenant() {
  http.get(`${BASE}/t/acme-enterprise/export`);
}

Each tenant group runs as its own scenario, and the thresholds filter on the scenario tag, so the run fails when the small tenants break their limit regardless of how the heavy tenant does.

The k6 thresholds documentation says that in k6, tagged requests create sub-metrics that you can use in thresholds.

The same k6 page says a failed threshold makes k6 exit with a non-zero exit code; in my runs, that code was 99.

I ran three 60-second configurations and restarted the server before each one. The CSV output records the status code of every request, which is how the results below separate 503 from 429:

# run 1: small tenants alone
PORT=3111 ISOLATION=off node server.mjs &
k6 run --summary-mode=full --out csv=requests-1.csv -e BASE_URL=http://127.0.0.1:3111 -e HEAVY_RATE=0 tenants.js

# run 2: add the heavy tenant, one shared queue (restart the server first)
PORT=3111 ISOLATION=off node server.mjs &
k6 run --summary-mode=full --out csv=requests-2.csv -e BASE_URL=http://127.0.0.1:3111 -e HEAVY_RATE=100 tenants.js

# run 3: add the heavy tenant with the per-tenant cap (restart the server first)
PORT=3111 ISOLATION=cap node server.mjs &
k6 run --summary-mode=full --out csv=requests-3.csv -e BASE_URL=http://127.0.0.1:3111 -e HEAVY_RATE=100 tenants.js

Results by Tenant

Run 1 is the baseline for the small tenants alone. Run 2 adds the heavy tenant with one shared queue, and run 3 adds it with the per-tenant cap. Percentiles and failure rates come from the k6 end-of-test summary; status codes come from the CSV output.

This is the threshold section and the small-tenant block of the k6 summary from run 2:

  █ THRESHOLDS

    http_req_duration{scenario:small_tenants}
    ✗ 'p(95)<150' p(95)=2.02s

    http_req_failed{scenario:small_tenants}
    ✗ 'rate<0.01' rate=5.95%

  █ SCENARIO: small_tenants

    http_req_duration..........: avg=1.71s   med=2s p(95)=2.02s    p(99)=2.03s    max=2.04s
    http_req_failed............: 5.95%  141 out of 2369
    http_reqs..................: 2369   38.18227/s
    dropped_iterations.........: 32     0.515759/s

time="2026-10-26T21:06:30+05:30" level=error msg="thresholds on metrics 'http_req_duration{scenario:small_tenants}, http_req_failed{scenario:small_tenants}' have been crossed"

• One queue made every tenant wait - in run 2 the small tenants' median response time was 2 s, right at the wait timeout, and k6 dropped 32 small-tenant iterations, which its metrics reference counts as iterations not started for lack of VUs.
• The cap moved the failures to the tenant causing them - in run 3 the small tenants matched their baseline, and the heavy tenant was served at about 25 requests per second, which is 2 connections divided by the 80 ms export.
• The error code changed - without isolation the small tenants got 503s when their wait timed out; with the cap, only the heavy tenant got errors, and they were 429 responses with a Retry-After header its client can act on.
• A blended threshold would have failed the healthy run - overall http_req_failed was 53.94% in run 3, because the heavy tenant's 429s made up most of its 8,401 requests, while the thresholds scoped to the small-tenant scenario passed.

Pool size, query times and tenant rates will differ in your system, so rerun both files with your own values before you rely on any of these limits.

How to Load Test a SaaS Application

The experiment isolates one failure mode. A full SaaS load testing plan covers the other shared resources too, in this order:

• Model the tenant mix by tier - pull per-tenant request rates from production logs, group tenants by plan, and give each tier its own k6 scenario or JMeter thread group with its own rate and endpoints.
• Set a limit per tier - write a p95 and an error-rate threshold for each tier, so the run fails when any tier breaks its own limit rather than when the blend does.
• Seed realistic tenant data - size test tenants like your largest real customers, since an empty test tenant answers queries from tiny tables and hides the slow ones.
• Test the rate limiter as a feature - drive one tenant past its limit and confirm it gets 429 with a Retry-After header while the others stay within their limits. API-heavy products apply the same tenant mix to API load testing.
• Run background work during the test - start exports, webhooks and scheduled jobs inside the load window, and watch queue depth and the age of the oldest job next to API latency.
• Hold the load through a scale-out - ramp faster than your autoscaler reacts and keep the load on until new capacity serves traffic, so you measure the gap between the burst and the new pods.
• Keep a small tenant-mix test in CI - a short version with per-tier thresholds fails the build when a change breaks isolation, before any customer notices.

To pick a load generator for this, compare the load testing tools that support scenarios or thread groups per tier; for a first k6 project, follow this k6 testing tutorial. Broader platform choices for SaaS teams are covered in this roundup of SaaS testing tools.

SaaS Performance Testing Metrics

Collect these per tenant or per tier as well as in total, since a noisy neighbor can leave the totals looking normal:

Running Tenant-Mix Tests at Scale

One machine is enough for a mock like the one above, but a production-sized tenant mix needs more load generators than one machine can drive. TestMu AI's cloud load testing, Performance Testing by HyperExecute, runs the JMeter and Gatling plans you already maintain on managed load generators, without a rewrite.

• Tenant data per generator - upload a JMeter plan with a CSV of tenant IDs and tiers, and the Split CSV option spreads the rows across threads and regions so generators do not all replay the same tenants.
• Regions - load can come from six regions with a percentage split, which suits tenants spread across geographies; multi-region load generation is listed for the Enterprise plan.
• Service and load in one job - on the Gatling YAML path, a background command can start the service under test on the same runner, so a mock like server.mjs and its load run in one job.
• k6 scripts - HyperExecute also runs k6 through the runtime addons block of its YAML, and the k6 example in its documentation pins k6 v0.52.0, so set the version your script needs.

Note: Run your tenant-mix JMeter and Gatling plans on managed cloud load generators, with no load-generation fleet to maintain between tests. Try TestMu AI free

Conclusion

Copy server.mjs and tenants.js, point the k6 script at a staging copy of your service, and set the small-tenant thresholds to the limit you promise your smallest plan. If your version of run 2 fails and run 3 passes, isolation works; if both fail, fix the shared pool before you add capacity.

When one machine cannot generate the tenant mix you need, run it on TestMu AI HyperExecute, starting from the HyperExecute k6 documentation.