> ## Documentation Index
> Fetch the complete documentation index at: https://qodex.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Load testing
> Put an API scenario under load from up to three regions, and read every number on the Load tests page, from p95 and error rate to what "Thresholds breached" means.
# Load testing
A load test sends many simulated users at your API at the same time, for a set length of time, and measures how fast and how reliably it answers. Qodex builds the load test from an API scenario you already have, runs it on [k6](https://k6.io) from up to three regions, and tells you whether your API held up.
This page explains how to start a run, what every number and label on the **Load tests** page means, and what to do when a run does not pass.
**Load tests** appears under **Test** in the sidebar only when load testing is switched on for your project. If you do not see it, ask your Qodex contact to turn it on.
## The short version
A run **passes** when, in every region it ran from, all three of these held:
| Threshold | Plain meaning |
| - | - |
| **p95 \< 1000ms** | 95 out of every 100 requests got a response in under one second. |
| **errors \< 1%** | Fewer than 1 in every 100 requests failed. |
| **checks > 99%** | More than 99 in every 100 of your scenario's checks passed, such as "the status is 200". |
If any one of them fails in any region, the run is marked **Thresholds breached**. That is not a crash or a Qodex error. It means the load test ran to the end and your API was slower, or failed more often, than the limit allows under that much traffic.
## Before you start
A load run needs three things. When you have none yet, the page lists them with **Done** or **To do** beside each.
1. **An API scenario.** Load tests replay API scenarios only, not UI scenarios. Save one from chat or on the **Scenarios** page.
2. **An environment.** The run sends its traffic to the base URL and variables of the environment you pick. Add one on the **Environments** page.
3. **A run left in your allowance.** The line under the page title shows your plan and how many load runs you have used. See [Your load run allowance](#your-load-run-allowance).
A load test sends real traffic to the environment you choose. Point it at a staging or test environment you are allowed to load, not at production, unless you mean to.
## Start a run
Click **Start new run**, then fill in:
| Field | What it means |
| - | - |
| **Scenario** | The API scenario every simulated user repeats. |
| **Environment** | Where the traffic goes. |
| **Concurrent users** | How many simulated users run the scenario at the same time, in **each** region. 1 to 100. |
| **Duration (sec)** | The **hold** time: how long the full number of users keeps going. 10 to 1800 seconds. |
| **Regions** | Where the traffic comes from: **US** East (N. Virginia), **UK**, EU West (London), and **SG**, Asia Pacific (Singapore). Pick one or more. |
The box under the fields shows the thresholds the run will be judged on (**Passes when** p95 \< 1000ms, errors \< 1%, checks > 99%), how long the run will take, and how much of your allowance it uses. Click **Run in N regions** to start.
**Run again** on a past run starts a new one with the same scenario, environment, users, duration, regions and thresholds.
## How a run works
Every run has three phases:
1. **Ramp up (30 s).** Users are added gradually, from zero up to your **Concurrent users** number, so your API is not hit by everyone at once.
2. **Hold.** All the users keep running for the **Duration** you set.
3. **Ramp down (30 s).** Users are removed gradually back to zero.
So a run with a 120-second duration takes about 3 minutes in total: 30 s up, 120 s hold, 30 s down.
Each simulated user (k6 calls this a **virtual user**) works in a loop. It runs every step of your scenario in order, checks each response, waits half a second, then starts the scenario again from the first step. Values a step captures, such as a token or an id, carry to later steps in the same pass through the scenario.
Each region runs the full number of users on its own. Fifty users across three regions is 150 users hitting your API at once.
## What the numbers mean
### Response time: p50, p90, p95, p99
Every request takes a certain time to get its response back. Qodex lines up all of a region's response times from fastest to slowest and reports these points along that line:
| Label | Meaning |
| - | - |
| **min** | The fastest request. |
| **p50** | Half the requests were faster than this. Also called the median. It shows a typical request. |
| **p90** | 90% of requests were faster than this. |
| **p95** | 95% of requests were faster than this. Only 1 in 20 was slower. |
| **p99** | 99% of requests were faster than this. Only 1 in 100 was slower. |
| **max** | The slowest request. |
**Why p95 and not the average?** An average hides slow requests. If 19 requests take 100ms and one takes 5 seconds, the average is about 350ms, which looks fine, yet one in twenty of your users waited 5 seconds. The p95 shows that slow edge, which is what real users notice. That is why the pass rule is written on p95.
**Example.** A p95 of 840ms means 95% of requests came back in under 840ms and 5% took longer. With a threshold of 1000ms, this region held its p95 threshold.
Times are written in milliseconds (`412ms`) or, from one second up, in seconds (`1.40s`).
### Requests and RPS
| Label | Meaning |
| - | - |
| **Requests** | How many HTTP requests the region sent during the whole run, ramps included. A scenario with 3 steps sends 3 requests each time a user goes through it. |
| **RPS** | Requests per second: **Requests** divided by the run's full length, ramps included. It is lower than your peak rate, because the ramps send less traffic than the hold. |
### Error rate
The **Error rate** is the share of requests that failed. A request counts as failed when:
* it got **no response** at all (the host could not be reached, the connection was refused, or the secure connection could not be set up), or
* it got a response with a status of **400 or higher** (4xx or 5xx).
Shown as a percentage: `0.40%` means 4 out of every 1,000 requests failed.
A status of 400 or higher counts as a failed request even when your scenario expects it. A step that checks "the status is 404" passes its check but still adds to the error rate. If such steps make up much of your scenario, the errors \< 1% threshold will not hold. Load test the steps that should succeed.
### Checks rate
Every step of your scenario has checks: its expected status code plus any assertions you added, such as "the body contains an `id`". The **Checks rate** is the share of all those checks, across every request in the run, that passed.
A request can get a fast `200` and still fail a check, for example when the body is missing a field under load. A falling checks rate is a sign your API starts returning wrong answers when it is busy, not just slow ones.
### Status codes
The **Status codes** row counts the responses by class:
| Class | Meaning |
| - | - |
| **2xx** | Success. |
| **3xx** | Redirect. |
| **4xx** | The request was refused or not found: for example 401 (not signed in), 404 (not found), or 429 (too many requests, often a rate limit). |
| **5xx** | The server failed: for example 500 (internal error), 502 or 503 (the server or a proxy in front of it is overloaded or down), or 504 (timed out upstream). |
Many 429s usually mean a rate limit in front of your API is turning away the load test's traffic. Many 5xx under load usually mean the API or its database ran out of capacity.
## Thresholds
The three thresholds are the pass rule. Each one is checked separately in each region, and the result is shown on the run's page as a chip:
| Chip | Meaning |
| - | - |
| ✓ (green) | The threshold held. |
| ✗ (amber) | The threshold was **breached**: the value went past the limit. |
| **not measured, no responses** (grey) | No request got a response, so there was no response time to compare. This is neither a pass nor a breach. |
The chips use k6's own names:
| Chip text | Threshold | Breached when |
| - | - | - |
| `http_req_duration: p(95)<1000` | p95 \< 1000ms | The region's p95 was 1000ms or more. |
| `http_req_failed: rate<0.01` | errors \< 1% | 1% or more of the region's requests failed. |
| `checks: rate>0.99` | checks > 99% | 1% or more of the region's checks failed. |
Runs started from the page use these defaults. **Run again** keeps the thresholds of the run it repeats.
## Run statuses
A run has one status for the whole run, worked out from its regions.
| Status | Meaning |
| - | - |
| **Queued** | Waiting to start. |
| **Running** | Sending traffic. |
| **Stopping** | A region was asked to stop and has not confirmed it yet. It may still be sending traffic. |
| **Passed** (on a region) / **All passed** (on the run) | Every region held all three thresholds. |
| **Thresholds breached** | The run finished, and every region breached at least one threshold. Your API handled the traffic, but too slowly or with too many failures. |
| **Mixed results** | The regions finished differently: for example one passed and another breached, or one breached and another errored. Open the run to see each region. |
| **Error** | The run could not produce a result. No numbers were measured, or not enough to judge. Hover the pill, or open the run, to see the reason. |
| **Cancelled** | Someone stopped the run before it finished. Hover the pill to see when. |
**Thresholds breached is about your API. Error is about the run.** A breached run measured your API and found it over a limit. An errored run never got a result, for example because the scenario could not be turned into a load test or the region could not start, so there is nothing to judge.
## The Load tests page
### The usage line
Under the page title, for example *Startup · 1 of 1 load run this month · resets Nov 1*. It shows your plan, how many load runs you have used in the current window, and when the next one becomes available. Hover it for which runs count.
### The five tiles
| Tile | What it shows |
| - | - |
| **Runs used** | Load runs used out of your allowance in the current window. |
| **Passed** | How many finished runs passed, out of the finished runs on the list, for example `3 of 5`. Cancelled, errored and unfinished runs are left out, and the tile says how many errored runs it left out. |
| **Latest p95** | The p95 of the newest finished run, taken from its **slowest region**, with how much it changed against the previous run of the same scenario (for example `↑ 120ms vs its previous run`). |
| **Error rate** | The error rate of the newest finished run, from the region with the highest one, next to the threshold. |
| **Peak load** | The most simulated users any run on the list sent at once: users per region × regions that sent requests, for example `50 × 3 regions` = `150 users`. |
The tiles count the recent runs on the list, not every run the project ever made.
### p95 across recent runs
A bar for each recent finished run, oldest on the left. Each bar is that run's slowest-region p95. A grey bar held the run's p95 threshold; an amber bar went over it. A rising line of bars means your API is getting slower from run to run.
### Scenarios under load
One row per scenario with its newest run: the **Last result**, its **p95**, how many **Runs** of it are on the list, and **Run again**.
### Recent runs
| Column | Meaning |
| - | - |
| **Started** | When the run started. Hover for the exact time. |
| **Scenario** | The scenario under load. |
| **Regions** | The regions it ran from. Hover a region for its own status. |
| **Users** | Concurrent users per region. |
| **Duration** | The hold time. |
| **Requests** | Requests sent, all regions added together. |
| **Worst p95** | The p95 of the slowest region. The run is judged on it, because a run passes only when every region holds. |
| **Worst error rate** | The highest error rate of any region. |
| **Status** | The run's status. A **Not charged** label beside it means the run did not use your allowance. |
A cell that reads **No responses** means requests were sent and none got an answer, so there is no response time to show. See [No responses](#no-responses).
## The run page
Click any run to open it. The top line shows the users, the hold and ramp, the number of regions, when it started and how long it took. **Cancel** stops a run in progress; **Run again** repeats a finished one.
Below that is one panel per region with:
* the region's own status;
* **Requests**, **RPS**, **p95**, **p99**, **Error rate** and **Checks rate**;
* **Latency**: bars for min, p50, p90, p95, p99 and max, so you can see how far the slow end stretches;
* **Status codes**;
* **Thresholds**: a chip for each of the three;
* **Error**: the reason, when the region errored;
* **Full report**: k6's own detailed report for that region, opened in a new tab.
Under the panels, **Recent runs of this scenario** lists earlier runs so you can compare them.
## No responses
**No responses** means the run sent requests and not one of them got an HTTP answer. That almost always means the region could not reach your API: the base URL is wrong, the host is private or behind a firewall or VPN, or it only allows traffic from certain IP addresses.
In that case there is no response time at all, so Qodex shows **No responses** instead of a misleading `0ms`, and the p95 threshold reads **not measured, no responses** rather than passed.
To fix it, check the environment's base URL, and make sure the API can be reached from the public internet in the regions you picked. The **Full report** shows the exact connection errors.
## Your load run allowance
Your plan includes a number of load runs per window (for example per month). The usage line and the **Runs used** tile show where you are.
* A run counts **once**, however many regions it uses.
* A run that never started, or that was cancelled or errored within its first minute without sending any measured load, does not count. It shows **Not charged** beside its status.
* A run cancelled or errored after that does count, because the load was already sent.
* When the allowance is used up, **Start new run** is turned off and says when the next run is available.
If your plan includes no load runs, the page says so and offers **Request upgrade**.
## Common questions
No. The test ran to the end and measured your API. At least one region went past a limit: p95 at 1000ms or more, 1% or more of requests failed, or 1% or more of checks failed. Open the run and look at the **Thresholds** chips in each region: the amber ✗ shows which limit was crossed.
The p95 threshold is about speed, not success. If 5% or more of requests took a second or longer, the p95 threshold is breached even when every response was a 200. Look at the **Latency** bars: p95 at or above 1.00s is the cause.
Three common causes. Under load your API may return 429 (rate limited) or 5xx (overloaded): check **Status codes**. A step that expects a 4xx status still counts as a failed request (see [Error rate](#error-rate)). Or the run cannot reach the host at all: see [No responses](#no-responses).
Each region measures from its own location. A region far from your servers adds network distance to every request, so its p95 is higher. The page judges the run on its slowest region, because your users in that part of the world see that time.
**Thresholds breached** means the run measured your API and it was over a limit. **Error** means the run could not produce a result at all, so nothing was judged. Hover the **Error** pill, or open the run, to see why.
The regions finished differently, for example one passed and another breached, or one breached and another errored. Open the run to see each region's panel.
RPS is averaged over the whole run, including the 30-second ramps when fewer users are active. Each user also waits half a second before going through the scenario again, and a slow API means each user gets through fewer requests.
Runs started from the page use the defaults: p95 \< 1000ms, errors \< 1%, checks > 99%. **Run again** keeps the thresholds of the run it repeats.
## Related
Build the API scenario a load test replays.
Run scenarios once, on a schedule, or from CI.
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.