Benchmarks
Current benchmark numbers are informal architecture checks, not claims of production capacity.
Repeatable cross-broker workload
Section titled “Repeatable cross-broker workload”The repository includes a shared Rust benchmark harness for Fibril, NATS JetStream and RabbitMQ. It supports fixed offered rate or saturation, configurable payload size, confirmation and delivery latency, memory/CPU samples, and message identity and settlement checks. Docker Compose provisions fresh single-node instances on the same selected filesystem. Each run retains its exact settings, logs, versions and result table.
The first profile compares one durable queue and one stored copy. JetStream timed-sync results carry a separate durability label. Stream fan-out and replicated cluster profiles are planned extensions.
Early observations
Section titled “Early observations”Internal measurements with the current TCP transport and durable path on a single Ubuntu node have observed:
| Workload | Observation |
|---|---|
| Ingress | roughly 250k+ messages/sec |
| Egress | roughly 250k+ messages/sec |
| Payload | 1KB messages |
| Machine | Ryzen 5950X |
Memory usage during these runs ranged from a few hundred MB at lower load to roughly 1-2GB near peak throughput, depending on queue depth, batching, and inflight state.
What the durability claim means
Section titled “What the durability claim means”Default queue publish confirmations wait for both the message and its enqueue state to cross the persistence boundary through fsync. Replicated durable policies additionally wait for persistence on the required replicas. The confirmation policy determines that requirement.
The front-page throughput figures describe the durable queue path on one node. The historical writer submits messages without awaiting a confirmation for each publish, so those figures are not a measurement of individual publish-confirmation throughput. A confirmed-publish rate needs a run that measures completed confirmations with a documented outstanding-confirmation window.
The historical runner can provision fresh solo or three-copy brokers for the shared Rust confirmation workload as well:
cargo build --release --bin fibril-servercargo build --release --locked --manifest-path benchmarks/comparison/Cargo.tomlpython3 scripts/bench-e2e-history.py \ --compare-bin benchmarks/comparison/target/release/broker-compare \ --output ./benchmark-results/confirmed-01 --storage /path/to/benchmark-data \ --copies 1 3 --clients 1 --prefetch 2000 --confirm-window 4096 \ --size 1024 --warmup-secs 5 --duration-secs 30This measures pipelined confirmations under saturation. Add --rate-per-sec
for a fixed offered rate. Confirmation credit and consumer prefetch are totals
shared across the connections in this mode. The client checks payload identity
and every confirmation, then the runner verifies final settlement on every copy.
confirmed.json retains confirmation and delivery latency, observed completion
rates and cohort throughput including drain. Resource boundary samples cover
workload launch through final settlement. The three-copy configuration uses
majority_durable. All copies share the selected filesystem unless
--copies 3 --node-storage ROOT0 ROOT1 ROOT2 selects their individual locations.
Comparison mode defaults to one connection pair. For a matched Fibril-only
confirmed/unconfirmed experiment, set --compare-delivery-window 32768 and
--confirm-window 32768 in both runs, then add --compare-unconfirmed to the
second run with a fresh output directory. Keep other settings identical. The
common delivery window bounds issued records until consumer ACK submission.
Confirmed mode also retains bounded outstanding confirmation credit.
Both modes finish with one confirmed barrier record per publisher connection, outside the timed cohort. All data and barrier records must be delivered uniquely and settled. The unconfirmed result reports a null confirmation rate and counts only those final barriers as confirmed. This isolates publish modes within the same harness and admission bounds. Historical mode retains ten connections for reproducing the older workload.
Bursty arrivals
Section titled “Bursty arrivals”The shared Rust harness supports repeating bursts with --burst-pattern 200,100.
For example, --rate 100 --warmup-secs 6 --duration-secs 60 offers an average of
100 messages/s through alternating 200-message and 100-message bursts. All
messages in a burst share their intended arrival time, preserving any client
waiting in the intended-to-delivery latency. Credit limits still apply.
Warmup and measurement must each contain whole burst cycles. Results include
per-message latency and the time until every message in each measured burst has
been delivered. This burst metric counts bursts rather than messages. The native
runner passes through --burst-pattern alongside --rate-per-sec and
--compare-bin. Keep burst and evenly spaced results separately labelled.
Interpretation
Section titled “Interpretation”These numbers are useful mostly as a sanity check:
- the durable path is not obviously too slow
- batching and the queue execution model are promising
- memory behavior still needs tuning
- larger payloads will shift bottlenecks toward memory, copying, storage, and network I/O
Missing benchmark work
Section titled “Missing benchmark work”The project still needs:
- broader payload-size sweeps across more hardware
- durability-setting comparisons
- longer repeated runs using the shared harness and client-headroom sweeps
- restart/replay timing
- multi-consumer fairness and backpressure scenarios
Historical two-process workload reproduction
Section titled “Historical two-process workload reproduction”scripts/bench-e2e-history.py preserves the original e2e_c writer and reader
as separate processes, with configurable connection count and prefetch. Build
the release binaries first, then select persistent result and storage directories:
cargo build --release --bin fibril-server --bin e2e_cpython3 scripts/bench-e2e-history.py --output ./benchmark-results/history-01 --storage /path/to/benchmark-dataThe default runs solo and three-copy majority sequentially, with 500,000 messages
per connection, ten connections and 1 KiB payloads. This is five million messages
in total. The older commands used five million per connection. For an equal
total with one connection, add --clients 1 --messages 5000000. The runner uses
fresh directories, checks replica readiness, keeps logs and resource samples,
and verifies final settlement. Data remains available after failures and success.
It requires Linux and Python 3.11 or newer for resource sampling and binary hashes.
Repository revisions and working-tree status are captured alongside binary hashes.
A 26 September 2026 SATA ext4 pass found the following with one connection and five million messages per row. All three replicated nodes shared the same drive and host. These are the initial single saturation runs, before the targeted wake-up fix, with default storage settings and no speculative delivery or early replication.
| Copies | Active delivery | Publish-to-delivery p50 / p99 | Sampled broker RSS peak |
|---|---|---|---|
| Solo | 232,428/s | 79 / 139 ms | 570 MiB |
| Three, majority durable | 90,752/s | 219 / 303 ms | 1,390 MiB total |
Both runs received the expected count with no observed retries, then reached zero ready/inflight and a settled frontier of five million on every copy. The writer uses unconfirmed submission, so its send rate is not durable-confirm throughput. Constant payloads and final counts do not establish unique delivery identity.
The initial ten-connection workload failed in both modes with a client heartbeat
timeout followed by BrokenPipe. Reducing solo prefetch to 1,024 did not resolve
it. Debugger stacks identified contention in ordered application, which woke all
pending operations on every completed range. Keratin now wakes only eligible
ranges during normal progress. Reset and failure still invalidate all waiters.
The unchanged ten-connection workload then passed with five million messages, no observed retries, and complete settlement on every copy:
| Copies | Active delivery | Publish-to-delivery p50 / p99 | Sampled broker RSS peak |
|---|---|---|---|
| Solo | 359,816/s | 620 / 1,009 ms | 650 MiB |
| Three, majority durable | 93,077/s | 2,125 / 2,599 ms | 1,137 MiB total |
The deeper saturation backlog raises latency. Single-connection solo repeats ranged from 223–232k/s before the fix and 209–244k/s afterward. These short runs do not establish a small performance difference. Historical SATA results were also reported above 250k/s, but the older OS and client prevent a controlled historical regression claim. The original ten-connection setup remains a regression workload.
This harness differs from the shared comparison harness, whose defaults include a 4,096 outstanding-confirmation window, 1,024 prefetch, timed warmup and measurement, and message-identity validation. Align those controls before comparing throughput across the harnesses.
Current TCP benchmark
Section titled “Current TCP benchmark”The current TCP-layer benchmark helper is e2e_c. It is still an early
benchmark, but it now reports wall throughput, active receive throughput, sent
and received counts, missing receive count, retry metadata observed on
delivered messages, and latency percentiles from publish time to reader
delivery.
For a quick local run, use the benchmark script:
MESSAGES=500000 CLIENTS=10 SIZE=1024 PREFETCH=16384 scripts/bench-e2e-c.shThe script builds the release server and benchmark binary, waits for
/healthz, runs a small warmup so lazy queue setup is out of the main
measurement path, then starts a reader and writer for the measured run.
Useful knobs:
| Variable | Default | Meaning |
|---|---|---|
MESSAGES | 500000 | Messages per client in the measured run |
CLIENTS | 10 | Parallel reader and writer client count |
SIZE | 1024 | Raw payload size in bytes |
PREFETCH | 16384 | Reader subscription prefetch |
WARMUP_MESSAGES | 1000 | Warmup messages before the measured run |
READY_SETTLE_SECONDS | 0.5 | Pause after reader readiness before starting the writer |
IDLE_TIMEOUT_MS | 10000 | Reader idle timeout before reporting partial receive counts |
CONFIRMED | 0 | Set 1 to wait for publish confirmations for correctness/debug checks |
LOG_FILE | temporary file | Build, server, and noisy runtime logs |
RESULTS_FILE | temporary file | Deterministic benchmark summary and queue snapshots |
The helper can also be run manually. Start the server in one terminal:
cargo run --release --bin fibril-serverStart the writer in another terminal:
cargo run --release --bin e2e_c -- -m 500000 -c 10 --writer --size 1024Start the reader in a third terminal, as close to the writer start time as practical:
cargo run --release --bin e2e_c -- -m 500000 -c 10 --reader --prefetch 16384The reader side prints latency percentiles when it receives messages. If the reader goes idle before receiving the target count, it reports the partial count and missing count instead of waiting indefinitely. The wall throughput includes any idle timeout tail, while active receive throughput uses the span between the first and last received message. Retry counts are read from Fibril’s reserved delivery metadata headers when present. Structured benchmark output and scenario tables are still future work.
Steady-state TCP benchmark
Section titled “Steady-state TCP benchmark”The burst benchmark intentionally lets writers run as fast as possible. That is useful for saturation checks, but it can build backlog and make latency look larger as the message count grows.
For latency at a controlled offered load, use the steady-state helper:
WRITERS=10 READERS=10 RATE_PER_SEC=100000 WARMUP_SECS=3 DURATION_SECS=10 \ SIZE=1024 PREFETCH=16384 scripts/bench-steady-c.shThe steady helper runs readers and writers in one coordinated process. It marks warmup messages separately, measures only the configured steady window, and prints both publish-to-delivery and server-receive-to-delivery latency. The wrapper also writes full server logs and full benchmark results to files, then prints a compact summary including publish and confirmation error counts plus server RSS average and peak sampled during the benchmark run.
The wrapper starts a local fibril-server on the default broker and admin
ports by default. Run one wrapper benchmark at a time. A second run will fail if
those ports are already occupied.
To target an already running server or a kept cluster, set START_SERVER=0,
BROKER_ADDR, and ADMIN_ADDR. DURABILITY_LABEL is only a result label, but
it is useful when comparing local, cluster-routed, and later replica-durable
confirmed runs.
To reuse the real cluster lifecycle checks, run the steady benchmark through the tryout script:
CONFIRMED=1 RATE_PER_SEC=1000 WARMUP_SECS=2 DURATION_SECS=5 \ scripts/cluster-tryout.sh --ganglion --nodes 3 --steady-benchcluster-tryout.sh starts the nodes, waits for the cluster, declares the
benchmark topic, runs the normal data-plane smoke, then calls
bench-steady-c.sh against the live cluster with START_SERVER=0. Use
--bench-topic <topic> when you want the tryout script to declare and benchmark
a different topic. This path currently measures clustered client routing. Treat
it as replica-durable only once the assignment and post-run follower state prove
the chosen follower applied the measured log range.
When CONFIRMED=1, writers still run with pipelined publish confirmations by
default. Set CONFIRM_WINDOW=1 if you specifically want the older serial
“publish, wait, publish” shape.
Useful knobs:
| Variable | Default | Meaning |
|---|---|---|
WRITERS | 10 | Parallel writer clients |
READERS | 10 | Parallel reader clients |
RATE_PER_SEC | 100000 | Target aggregate publish rate |
WARMUP_SECS | 5 | Warmup duration excluded from measured results |
DURATION_SECS | 30 | Steady measurement duration |
DRAIN_TIMEOUT_SECS | 10 | Time allowed for measured messages to drain |
SIZE | 1024 | Raw payload size in bytes |
PREFETCH | 16384 | Reader subscription prefetch |
CONFIRMED | 0 | Set 1 to require broker publish confirmations |
CONFIRM_WINDOW | 1024 | In-flight confirmations per writer when CONFIRMED=1 |
TOPIC | topic1 | Queue topic used by the steady benchmark |
START_SERVER | 1 | Set 0 to target an external server or cluster |
BROKER_ADDR | 127.0.0.1:9876 | Broker TCP address passed to the benchmark client |
ADMIN_ADDR | 127.0.0.1:8081 | Admin address used for health checks and queue snapshots |
DURABILITY_LABEL | local | Label printed into results and summary tables |
BUILD | 1 | Set 0 to skip rebuilding release binaries |
LOG_FILE | temporary file | Build, server, and noisy runtime logs |
RESULTS_FILE | temporary file | Deterministic benchmark summary and queue snapshots |
Memory numbers are sampled from the local fibril-server process RSS once per
second during the wrapper benchmark. The average is over the sampled run period,
not precisely only the warmup-excluded steady window.
For repeatable local sweeps, use the matrix helper:
scripts/bench-matrix.sh smokescripts/bench-matrix.sh baseline confirmedscripts/bench-matrix.sh throughput-1k payloadThe matrix helper builds the release server and benchmark binary once, then
runs named steady-state cases with one results file and one log file per case.
It also writes summary.md, a Markdown table generated from those result
files. Set OUT_DIR=... to choose where files go. Without arguments, it runs
only the quick smoke scenario.
Available scenarios:
| Scenario | Purpose |
|---|---|
smoke | Short low-rate sanity check |
baseline | 1KB 50k/s and 150k/s unconfirmed |
confirmed | 1KB 50k/s and 150k/s with pipelined confirmations |
throughput-1k | Higher-rate 1KB exploratory sweep |
payload | 8KB, 64KB, 512KB, and 1MB spot checks |
large-backlog | Large-payload cases expected to build backlog |
all | baseline, confirmed, throughput-1k, and payload |
To regenerate a table from existing result files:
scripts/bench-results-table.sh bench-results/steady-*/baseline-*.results.txtHow to read the numbers
Section titled “How to read the numbers”The offered rate is the requested publish rate, not a guarantee that the broker or machine can keep up without queueing. When actual measured rate reaches the target but latency climbs, the run is usually showing backlog, not low-latency capacity. When actual measured rate falls below target, writers or the machine could not sustain the requested input rate.
Measured missing should usually be zero. Non-zero values mean the benchmark
stopped before every measured message was delivered, commonly because the drain
timeout expired or the run failed.
RSS is sampled from the server process only. It excludes benchmark client memory and the operating system page cache, so it is useful for comparing local runs but not a full machine-memory budget.
Quick validation run from June 7, 2026, using WRITERS=10, READERS=10,
SIZE=1024, PREFETCH=16384, WARMUP_SECS=2, and DURATION_SECS=5:
| Mode | Target rate | Actual measured rate | Missing | publish→deliver p50/p95/p99/max | server-receive→deliver p50/p95/p99/max | Errors | End queue |
|---|---|---|---|---|---|---|---|
| unconfirmed | 50k/s | 50,000/s | 0 | 15 / 22 / 25 / 58 ms | 10 / 16 / 18 / 20 ms | 0 | ready=0, inflight=0 |
| unconfirmed | 150k/s | 149,999/s | 0 | 11 / 15 / 17 / 56 ms | 9 / 13 / 14 / 17 ms | 0 | ready=0, inflight=0 |
| confirmed, window=1024 | 50k/s | 50,010/s | 0 | 14 / 19 / 21 / 28 ms | 11 / 15 / 16 / 21 ms | 0 | ready=0, inflight=0 |
| confirmed, window=1024 | 150k/s | 149,999/s | 0 | 12 / 16 / 17 / 21 ms | 10 / 13 / 15 / 18 ms | 0 | ready=0, inflight=0 |
Higher-rate exploratory sweep from the same run shape:
| Mode | Target rate | Actual measured rate | Missing | publish→deliver p50/p95/p99/max | Notes |
|---|---|---|---|---|---|
| unconfirmed | 250k/s | 250,017/s | 0 | 13 / 16 / 17 / 55 ms | Clean short run |
| unconfirmed | 350k/s | 350,002/s | 0 | 79 / 114 / 122 / 130 ms | Latency knee starts showing |
| unconfirmed | 400k/s | 400,000/s | 0 | 792 / 1060 / 1096 / 1107 ms | Drains, but backlog-driven |
| unconfirmed | 450k/s | 449,953/s | 0 | 1807 / 2038 / 2055 / 2066 ms | Drains, high latency |
| unconfirmed | 500k/s | 499,955/s | 0 | 2520 / 2783 / 2815 / 2833 ms | Drains, high latency |
| unconfirmed | 600k/s | 599,916/s | 0 | 3806 / 4546 / 4579 / 4600 ms | Drains, very high latency |
For this short local run, the practical low-latency region is below roughly 350-400k/s for 1KB messages. Above that, the broker can still drain the run, but latency reflects backlog building during the measurement window.
Pipelined confirmed publishes follow the same pattern. With
CONFIRM_WINDOW=1024, a 400k/s target reached about 385k/s. Raising the window
to 4096 reached the 400k/s target, and 450k/s also reached target, but latency
rose into the 1-2 second range. Larger windows are useful for saturating the
path while preserving publish confirmation correctness. They are not a latency
optimization.
Payload-size spot checks on the same SATA SSD development machine:
| Payload | Target rate | Actual measured rate | Missing | publish→deliver p50/p95/p99/max | Server RSS avg/peak | Notes |
|---|---|---|---|---|---|---|
| 8KB | 50k/s | 50,010/s | 0 | 14 / 17 / 19 / 61 ms | not sampled | Clean short run |
| 8KB | 150k/s | 139,987/s | 0 | 2608 / 3117 / 3168 / 3245 ms | not sampled | Could not reach target, backlog-driven |
| 64KB | 10k/s | 10,000/s | 0 | 18 / 22 / 23 / 32 ms | not sampled | Clean short run |
| 64KB | 20k/s | 18,285/s | 0 | 1605 / 1891 / 2144 / 2277 ms | not sampled | Could not reach target, likely storage-bandwidth bound |
| 512KB | 1k/s | 999/s | 0 | 27 / 34 / 39 / 47 ms | 262.9 / 310.2 MiB | Clean short run |
| 512KB | 2k/s | 2,000/s | 0 | 1165 / 1669 / 1756 / 1841 ms | 951.4 / 1538.0 MiB | Drains, but backlog-driven |
| 1MB | 500/s | 498/s | 0 | 33 / 45 / 51 / 63 ms | ~290 / ~334 MiB | Clean short run. Reruns varied slightly |
| 1MB | 1k/s | 1,000/s | 0 | 1812 / 2539 / 2693 / 2801 ms | 847.0 / 1187.5 MiB | Drains, but backlog-driven |
For larger payloads, the bottleneck shifts away from message scheduling and toward memory copying, TCP throughput, and especially durable storage bandwidth. On this SATA SSD machine, 64KB at 10k/s is already roughly 625 MiB/s of application payload before protocol, replication within the durable path, and filesystem overhead. Treat payload-size numbers as hardware-specific. The large-payload memory samples also show the expected split: clean runs can stay in the low hundreds of MiB, while backlog-driven runs retain much more payload data in-process and can exceed 1 GiB RSS.
Recent local exploratory run, using WRITERS=10, READERS=10, SIZE=1024,
PREFETCH=16384, WARMUP_SECS=3, and DURATION_SECS=10:
| Target rate | Actual measured rate | Missing | publish→deliver p50/p95/p99/max | server-receive→deliver p50/p95/p99/max | End queue |
|---|---|---|---|---|---|
| 50k/s | 49,962/s | 0 | 17 / 25 / 29 / 63 ms | 11 / 17 / 19 / 52 ms | ready=0, inflight=0 |
| 100k/s | 99,785/s | 0 | 13 / 18 / 62 / 136 ms | 10 / 14 / 57 / 130 ms | ready=0, inflight=0 |
| 150k/s | 149,831/s | 0 | 12 / 17 / 173 / 225 ms | 10 / 14 / 171 / 222 ms | ready=0, inflight=0 |
| 200k/s | 199,733/s | 0 | 13 / 78 / 259 / 294 ms | 11 / 76 / 258 / 292 ms | ready=0, inflight=0 |
| 250k/s | 249,591/s | 0 | 14 / 260 / 367 / 397 ms | 12 / 258 / 365 / 394 ms | ready=0, inflight=0 |
| 300k/s | 298,528/s | 0 | 18 / 578 / 623 / 659 ms | 16 / 577 / 613 / 655 ms | ready=0, inflight=0 |
These older results remain useful as a development checkpoint, but the newer short sweeps above are a better current summary: the broker can drain substantially higher short-run rates, while the practical low-latency region depends heavily on payload size, durable storage bandwidth, and whether backlog is allowed to build. Treat all tables here as reproducible local checkpoints, not capacity promises.
Plexus stream benchmarks
Section titled “Plexus stream benchmarks”Plexus streams fan out: every live subscriber receives every matching record, so
delivered throughput is readers x published. Streams have three durability
tiers (durable, speculative, ephemeral) that trade the delivery and producer
confirm timing against the storage guarantee. The helper is
scripts/bench-stream.sh, the stream counterpart of bench-steady-c.sh:
DURABILITY=ephemeral RATE_PER_SEC=150000 WRITERS=4 READERS=2 \ CONFIRMED=1 CONFIRM_WINDOW=4096 scripts/bench-stream.shDURABILITY selects the tier and DATA_DIR selects the data filesystem, so the
same run can be compared on an SSD versus a tmpfs to isolate storage effects.
Durability tiers
Section titled “Durability tiers”1KB records, four writers, two fan-out readers, one partition, pipelined confirmations (window 4096), on the same SATA SSD development machine. Latency is publish to delivery:
| Tier | Target rate | Measured rate | deliver p50/p95/p99/max | Confirm p50 | RSS peak |
|---|---|---|---|---|---|
| durable | 50k/s | 49,952/s | 7 / 9 / 27 / 56 ms | 10 ms | 53 MiB |
| speculative | 50k/s | 50,000/s | 1 / 2 / 3 / 10 ms | 10 ms | 65 MiB |
| ephemeral | 50k/s | 49,986/s | 1 / 2 / 3 / 9 ms | 4 ms | 49 MiB |
| durable | 150k/s | 149,597/s | 49 / 67 / 95 / 128 ms | 50 ms | 103 MiB |
| speculative | 150k/s | 149,653/s | 2 / 16 / 36 / 57 ms | 55 ms | 140 MiB |
| ephemeral | 150k/s | 149,800/s | 1 / 2 / 10 / 28 ms | 2 ms | 78 MiB |
| durable | 250k/s | 235,306/s | 64 / 82 / 97 / 108 ms | 64 ms | 137 MiB |
| speculative | 250k/s | 236,722/s | 3 / 19 / 28 / 59 ms | 45 ms | 155 MiB |
| ephemeral | 250k/s | 249,256/s | 7 / 31 / 45 / 53 ms | 8 ms | 124 MiB |
The tiers separate as designed:
durablewaits for the fsync before it delivers and confirms, so its delivery latency is the fsync latency. Strictest guarantee, highest and most predictable latency.speculativedelivers as soon as the record is staged and defers the producer confirm until it is durable. Delivery is near-instant while the confirm reflects real durability, and records carry afibril.speculativeheader so a consumer knows they may still be rolled back.ephemeraldelivers and confirms at staging and persists in the background. Lowest latency on every axis and the lightest on memory. It keeps a tight tail on a real disk because a background flush drains dirty pages on keratin’s fsync worker stage rather than letting them pile up until the kernel throttles the writer.
Fan-out scaling
Section titled “Fan-out scaling”Every reader is an independent subscriber that receives the whole stream, so the
delivered rate is readers x publish rate until a bottleneck bites. 1KB, 100k/s
offered, one partition, scaling the reader count.
Ephemeral tier, cursorless readers reading from the live tail:
| Readers | Publish rate | Delivered rate | Per reader | deliver p50/p95/p99/max | RSS peak |
|---|---|---|---|---|---|
| 1 | 99,990/s | 99,990/s | 99,990/s | 0 / 2 / 3 / 17 ms | 58 MiB |
| 2 | 99,990/s | 199,980/s | 99,990/s | 1 / 2 / 4 / 20 ms | 60 MiB |
| 4 | 100,006/s | 400,023/s | 100,006/s | 0 / 2 / 3 / 17 ms | 66 MiB |
| 8 | 99,984/s | 799,866/s | 99,983/s | 1 / 3 / 17 / 52 ms | 80 MiB |
| 16 | 97,165/s | 1,546,066/s | 96,629/s | 3 / 52 / 171 / 233 ms | 169 MiB |
| 32 | 93,687/s | 850,976/s | 26,593/s | 969 / 1175 / 1269 / 1387 ms | 206 MiB |
Durable tier, auto-ack readers (each commits a durable cursor per record):
| Readers | Publish rate | Delivered rate | Per reader | deliver p50/p95/p99/max | RSS peak |
|---|---|---|---|---|---|
| 1 | 100,000/s | 99,570/s | 99,570/s | 59 / 85 / 115 / 173 ms | 87 MiB |
| 2 | 100,000/s | 199,256/s | 99,628/s | 60 / 93 / 143 / 178 ms | 88 MiB |
| 4 | 100,000/s | 396,174/s | 99,043/s | 64 / 90 / 115 / 150 ms | 89 MiB |
| 8 | 99,990/s | 788,713/s | 98,589/s | 62 / 97 / 127 / 176 ms | 87 MiB |
| 16 | 99,832/s | 1,105,715/s | 69,107/s | 196 / 613 / 646 / 732 ms | 140 MiB |
| 32 | 95,410/s | 759,266/s | 23,727/s | 1067 / 1250 / 1360 / 1505 ms | 201 MiB |
Both tiers fan out near-linearly while a single partition’s fan-out actor has headroom: every reader sees the full 100k/s up to eight readers. The ephemeral tier peaks around 1.5M frames/s at sixteen readers, where the single per-partition fan-out actor and its delivery tasks saturate. Past that knee one partition thrashes, so thirty-two readers delivers less aggregate than sixteen at backlog-driven latency. The durable tier holds the same near-linear shape to eight readers and pays a steady delivery-latency floor for the fsync-before-deliver guarantee, with its per-reader cursor work bringing the knee in a little earlier. Durable auto-ack at this fan-out is only viable because cursor commits are microbatched, coalesced per partition into one durable record and one actor message per window. Committing a cursor per record inline collapsed delivery to tens of records per second per reader at multi-second latency.
Those two tables are bounded by delivery throughput: aggregate frames per second (readers times publish rate) saturating one partition’s fan-out actor. That is a separate question from how many readers a partition can fan out to when delivery throughput is not the bottleneck. Holding the rate low (1KB ephemeral, 1k/s offered) so the aggregate stays well under the ceiling, reader count scales cleanly:
| Readers | Publish rate | Delivered rate | Per reader | deliver p50/p95/p99/max | RSS peak |
|---|---|---|---|---|---|
| 32 | 1,000/s | 32,000/s | 1,000/s | 4 / 6 / 7 / 9 ms | 40 MiB |
| 64 | 1,000/s | 64,000/s | 1,000/s | 4 / 6 / 7 / 9 ms | 47 MiB |
| 128 | 1,000/s | 128,000/s | 1,000/s | 4 / 6 / 7 / 9 ms | 61 MiB |
| 256 | 1,000/s | 256,000/s | 1,000/s | 4 / 7 / 7 / 11 ms | 84 MiB |
All 256 readers on a single partition keep up at a flat few-millisecond latency, and memory grows gently with connection count. Fan-out reach is cheap: the limit is total delivered frames per second, not the number of readers, so one partition serves many readers as long as readers times rate stays under its delivery ceiling.
The lever past the single-partition delivery ceiling is partitions: each partition has its own fan-out actor and reader connections, so spreading a stream across partitions scales delivery throughput horizontally.
Reading these numbers
Section titled “Reading these numbers”These are reproducible local checkpoints, not capacity promises, the same as the
queue numbers above. The per-reader delivered rate matching the publish rate in
every row is how the bench shows each reader received the whole stream rather
than a thinned subset. ephemeral confirms before the record is durable (best
effort), so a crash can lose the most recent unflushed records. That is the
tradeoff for its latency, and the durable tier exists for when that is not
acceptable.