# tapper Load Tests

Load testing suite for the tapper matchmaking platform, built with [k6](https://k6.io/) plus one ApacheBench run for the explicit 1000-request requirement.

## Prerequisites

Install k6:

```bash
# macOS
brew install k6

# Windows (choco)
choco install k6

# Windows (winget)
winget install k6

# Linux (Debian/Ubuntu)
sudo gpg -k
sudo gpg --no-default-keyring --keyring /usr/share/keyrings/k6-archive-keyring.gpg \
  --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D68
echo "deb [signed-by=/usr/share/keyrings/k6-archive-keyring.gpg] https://dl.k6.io/deb stable main" \
  | sudo tee /etc/apt/sources.list.d/k6.list
sudo apt-get update && sudo apt-get install k6

# Docker
docker pull grafana/k6
```

Make sure the tapper stack is running (`docker compose up -d`) before executing tests.

## Test Scripts

| Script | Purpose | VUs | Duration |
|---|---|---|---|
| `smoke-test.js` | Quick sanity check | 2 | 30s |
| `script.js` | Full load test | 10 to 100 | ~4 min |
| `stress-test.js` | Find breaking point | 10 to 250 | ~3 min |
| `failover-test.js` | Resilience / instance kill | 50 | ~3 min |
| `quick-failover.js` | Live demo failover (kill api-2 at t+5s) | 20 | 20s |
| `ab-auth-profile.ps1` | ApacheBench against authenticated DB read | 50 | 1000 requests |

### ApacheBench: authenticated DB-read endpoint

This satisfies the 1000-request ApacheBench requirement. The script registers a real user, extracts the access token, then runs ApacheBench against `GET /api/profile`. That endpoint verifies the JWT access token and reads the current user from PostgreSQL.

```powershell
.\loadtest\ab-auth-profile.ps1
```

Defaults:

- Host registration URL: `http://localhost`
- ApacheBench container: `httpd:2.4-alpine`
- Docker network: `tapper-web`
- Target URL inside Docker: `http://traefik/api/profile`
- Load shape: `ab -n 1000 -c 50`

### Quick failover (presentation demo)

A 20-second variant of the failover test, designed for live presentation. One
user is pre-registered in `setup()` to skip per-iteration bcrypt cost, then 20
VUs hammer `GET /api/profile` for 20 seconds. Run it in one terminal and run
`docker stop tapper-api-2-1` at the 5-second mark in another.

```bash
docker run --rm --network tapper-web \
  -v "$PWD/loadtest:/scripts" \
  -e BASE_URL=http://traefik \
  grafana/k6:latest run /scripts/quick-failover.js

# in a second terminal, ~5 seconds in:
docker stop tapper-api-2-1
```

Expected: ~840 req/s sustained, ~96% success rate including the error blip
during Traefik's ~5-second failover detection window, p95 around 4ms on the
access-token-protected reads.

## Running Tests

All commands assume you are in the repository root.

### Smoke Test (run first)

```bash
k6 run loadtest/smoke-test.js
```

Verifies all endpoints are reachable. Run this after every deploy.

### Main Load Test

```bash
k6 run loadtest/script.js
```

Simulates realistic traffic ramping from 10 to 100 concurrent users. Tests the full user journey: register, login, view profile, check leaderboard, browse matches, and queue join/leave.

### Stress Test

```bash
k6 run loadtest/stress-test.js
```

Ramps up to 250 VUs to find the platform's breaking point. The test uses lenient thresholds and will only abort if the error rate exceeds 50%.

### Failover Test

```bash
# Terminal 1: start the test
k6 run loadtest/failover-test.js

# Terminal 2: at ~45 seconds, kill one API instance
docker stop tapper-api-2

# Terminal 2: (optional) at ~90 seconds, restart it
docker start tapper-api-2
```

Measures how the system handles losing an API instance. Traefik should route around the failure within ~10 seconds.

### Custom Base URL

All tests default to `http://localhost:3000`. Override with:

```bash
k6 run -e BASE_URL=http://your-host:3000 loadtest/script.js
```

### JSON Output (for analysis)

```bash
k6 run --out json=loadtest/results/output.json loadtest/script.js
```

## Expected Results

### Smoke Test

All checks should pass. If any fail, the API has a functional issue that must be fixed before proceeding to load tests.

### Main Load Test, Thresholds

| Metric | Threshold | Meaning |
|---|---|---|
| `http_req_duration` p95 | < 500 ms | 95% of requests complete in under 500ms |
| `http_req_failed` | < 1% | Less than 1% of requests return errors |
| `custom_profile_duration` p95 | < 300 ms | Profile endpoint is fast |
| `custom_leaderboard_duration` p95 | < 400 ms | Leaderboard query is efficient |
| `custom_register_duration` p95 | < 800 ms | Registration (includes bcrypt) |
| `custom_login_duration` p95 | < 600 ms | Login (includes bcrypt verify) |

### Stress Test, What to Look For

The stress test intentionally pushes past capacity. Key observations:

1. **Inflection point**: At which VU count does p95 latency spike above 1 second?
2. **Error onset**: When do HTTP errors first appear?
3. **Error type**: Are errors 502 (Traefik can't reach backend), 500 (server overloaded), or timeouts?
4. **Recovery**: After VUs ramp down, does the system stabilize?

### Failover Test, What to Measure

| Metric | Before Kill | During Kill | After Recovery |
|---|---|---|---|
| Throughput (req/s) | Baseline | ~50% drop | Return to baseline |
| Error rate | ~0% | Spike (expect 10-30%) | ~0% |
| p95 latency | Baseline | Higher | Return to baseline |
| Recovery time | N/A | N/A | < 15 seconds |

## Interpreting Results

### k6 Output Summary

After each test, k6 prints a summary. Key lines:

```
http_req_duration...: avg=45ms  min=2ms  med=30ms  max=2.1s  p(90)=120ms  p(95)=250ms
http_req_failed.....: 0.42%   ✓ 18     ✗ 4264
iterations..........: 4282    35.68/s
```

- **avg/med/p95**: Average, median, and 95th percentile response times
- **http_req_failed**: Percentage and count of failed requests
- **iterations**: Total completed iterations and throughput

### Threshold Pass/Fail

k6 marks thresholds with a check or cross:

```
✓ http_req_duration...: p(95)<500   → 250ms
✗ http_req_failed.....: rate<0.01   → 4.2%  ← FAILED
```

### Using JSON Output

For timeline analysis, export to JSON and use k6's built-in tools or third-party dashboards:

```bash
# Export
k6 run --out json=results.json loadtest/script.js

# Analyze with jq (example: errors over time)
cat results.json | jq 'select(.type=="Point" and .metric=="http_req_failed")'
```

For richer visualization, consider [Grafana + InfluxDB](https://k6.io/docs/results-output/real-time/influxdb-+-grafana/) or [k6 Cloud](https://k6.io/cloud/).

## Bottleneck Analysis Guide

### Expected Bottlenecks (from most to least likely)

| Bottleneck | Symptoms | Solution |
|---|---|---|
| PostgreSQL connection pool | Sudden error spike at a specific VU threshold; errors are 500s with "connection pool exhausted" | Increase `max_open_conns` in Go config, or add PgBouncer |
| JWT verification CPU | Gradual latency increase correlated with CPU usage | Switch to ES256 (ECDSA) or cache verified tokens |
| Single DB instance | Write contention on registration; lock waits | Add read replicas for leaderboard/profile queries |
| Traefik routing | 502 errors even when backends are healthy | Tune Traefik retry/circuit-breaker middleware |
| Network (Docker bridge) | Uniform latency increase across all endpoints | Use host networking or optimize Docker network |

### How to Diagnose

1. **Run the stress test** and note the VU count when errors begin.
2. **Check container resources** during the test:
   ```bash
   docker stats --no-stream
   ```
3. **Check PostgreSQL connections**:
   ```sql
   SELECT count(*) FROM pg_stat_activity;
   SELECT max_conn FROM pg_settings WHERE name = 'max_connections';
   ```
4. **Check API logs for errors**:
   ```bash
   docker logs tapper-api-1 --tail 50
   docker logs tapper-api-2 --tail 50
   ```

## File Structure

```
loadtest/
├── README.md            # This file
├── script.js            # Main load test (10 → 100 VUs)
├── smoke-test.js        # Quick sanity check (2 VUs, 30s)
├── stress-test.js       # Breaking point finder (10 → 250 VUs)
├── failover-test.js     # Resilience test (50 VUs + instance kill)
└── helpers/
    └── auth.js          # Shared auth helpers (register, login, JWT)
```
