# Performance Benchmark Plan

Status: **DOCUMENTED**. A reproducible harness lives in
`scripts/bench-receipt.ts` (partial: canonicalize / sha256 / ed25519 /
append phases covered; api / verify / chain phases marked TODO). No
numerical throughput claim is made outside a signed benchmark report.

## Rule

No public artefact — website copy, README, architecture page, docs,
sales material — may state a throughput number ("N receipts / second",
"N rps") unless it can point to a matching, dated, reproducible
benchmark report. The report must include the commit hash, hardware
profile, config, and full results.

Speculative numbers ("we can handle 1000 rps") are prohibited until
measured.

## Phases to measure independently

Each phase is measured **in isolation** so that ceilings are attributable.

| Phase | What is measured | Isolated benchmark hook |
|-------|------------------|-------------------------|
| Canonicalisation | `canonicalize(body)` for a representative body size distribution | `bench:canonicalize` |
| SHA-256 | Digest over the canonicalised bytes | `bench:sha256` |
| Ed25519 sign | Sign the receipt hash | `bench:sign` |
| Ledger append | Write one line to the tail with the atomic-lock cross-process safety in place | `bench:append` |
| Indexing | Update in-memory index for search / listing | `bench:index` |
| API/network | Full HTTP round trip on `POST /api/v1/receipts` | `bench:api` |
| Verification | `verifyReceipt` on a single receipt | `bench:verify` |
| Chain validation | Full-ledger integrity walk | `bench:chain` |

## Profiles

Each phase and the end-to-end path are exercised at multiple offered
load levels:

- 1 req/s
- 10 req/s
- 100 req/s
- 500 req/s
- 1000 req/s **only if the target hardware sustains it without loss**

Sustained load: at least 5 minutes at each level after warm-up.
Burst load: 3× the sustained level for 5 s, repeated.

If a level cannot be sustained, the report says so plainly; it does not
label a partial run as passing.

## Metrics matrix

Per phase and per profile:

- Throughput (achieved, not offered).
- Latency p50 / p95 / p99 / p99.9.
- Error rate, categorised (5xx / 4xx / retryable 503 / timeout).
- CPU %.
- Resident memory.
- Event-loop delay (Node `perf_hooks`).
- Disk write bytes and IOPS.
- Queue depth (in-flight seals, express queue).
- Recovery time after a controlled restart during load.

## Test topologies

- Single Node process (default).
- Multi-process on the same host (with `scripts/multiprocess-concurrency-test.ts`
  as the concurrency stressor).
- Restart under load (`scripts/chaos-restart-under-load.ts` combined
  with `bench:api`).
- Failed signing path (invalid input, corrupted state).
- Disk-full simulation on an isolated tmpfs — never on live storage.
- Lock contention at 4× concurrency of physical cores.

## Report format

Each benchmark run emits `bench-report-<utc>.json` with:

```jsonc
{
  "report_id": "bench-YYYYMMDD-HHMMSSZ",
  "commit": "<git-sha>",
  "generated_at": "<iso>",
  "hardware": {
    "os": "linux 6.x", "arch": "x86_64",
    "cpu_model": "...", "cores": 8, "memory_mb": 16384,
    "disk": "nvme-ssd"
  },
  "config": {
    "node_version": "20.x",
    "signatrust_version": "…",
    "profile": "single_process | multi_process | chaos",
    "data_dir": "/tmp/bench-…"
  },
  "phases": [
    { "phase": "sign", "throughput_rps": 4200, "p50_ms": 0.24, "p95_ms": 0.31, "p99_ms": 0.42, "err_rate": 0.0 }
  ],
  "notes": "…"
}
```

Reports are stored in `docs/architecture-alignment/benchmarks/` and are
the **only** source that public artefacts may quote. Every quoted number
must link back to a report file.

## Reproducibility

- Harness runs against a fresh temporary data directory.
- No live-service data is touched.
- The harness prints its exact commit hash on start and refuses to run
  if the working tree is dirty (`--allow-dirty` to override for local
  experimentation, which taints the report).
- Seeds for randomised payloads are recorded in the report so any run
  can be reproduced byte-for-byte.

## Anti-pattern: extrapolating from micro-benchmarks

Signing at 4000 ops/s in isolation does not mean the system sustains
4000 rps end-to-end. Bottlenecks stack: canonicalisation + sign + append
+ index + HTTP + client-side network + event-loop pressure + logging.
Only the end-to-end `bench:api` numbers may be quoted as
"receipts per second".

## What is NOT in the plan

- No load test against production.
- No comparison against unnamed competitors.
- No aspirational number that has not been measured on the exact commit
  it claims.

## Implementation status

`scripts/bench-receipt.ts` (added in this pass) covers:

- `bench:canonicalize`
- `bench:sha256`
- `bench:sign`
- `bench:append` (single-process; multi-process depends on
  `scripts/multiprocess-concurrency-test.ts` as a driver)

Marked TODO:

- `bench:api` end-to-end
- `bench:verify`
- `bench:chain`

These are open engineering items and are not blocked by protocol design.