← all runs

Cohere-100M — 2026-10-10

QPS at recall

Throughput under load

Each configuration is measured at its operating point for the target: its fastest sweep point whose recall reaches it. Values are measured there, not interpolated, so the recall can be above the target. Each cell is wall-clock QPS over the whole run, then in parentheses the throughput while every client is busy, which leaves out the drain at the end, and below them the p50 and p95 query latency over all clients. With more than one client, N× is the speedup: wall-clock QPS over the same configuration's single-client wall-clock QPS at that target. Ideal scaling matches the client count until the host's CPUs saturate, after which it flattens. Clients are concurrent connections from rekall, and CPU counts are the rekall host's. Wall-clock QPS includes rekall's own work between queries, so the 1-client row reads a little below QPS at recall, which comes from query latency. Recall is distance-based (ann-benchmarks, tie-aware).

Measured on config — prism-defaults @0.95: prism-defaults (r=0.962) · prism-defaults @0.99: prism-defaults (r=0.991)

Latency at recall

All values are taken at exactly the stated recall: each configuration's own sweep is log-interpolated between its two surrounding measured points (measured values are used verbatim on an exact hit) and the best configuration at that recall is shown — values are never interpolated across different configurations. “—” means no single sweep spans that recall; nothing is extrapolated. Recall is distance-based (ann-benchmarks, tie-aware). Insert/update/delete workloads are not measured.

Build comparison

Plots

Host

Versions

Search

Index builds

prism-defaults: {"boundary_epsilon":null,"centroid_compression":null,"centroid_fastscan":null,"fan_out":null,"fastscan":null,"fastscan_bits":null,"nlist":null,"rerank_vectors":null,"soar_lambda":null}

instance type          r8i.8xlarge
vCPU (observed)        32
memory (observed)      247.7 GiB
cpu model              Intel(R) Xeon(R) 6975P-C
kernel                 7.0.0-1014-aws
data directory         /var/lib/postgresql/18/main
mount point            /
filesystem             ext4
media                  ebs
device                 nvme0n1
device size            2147 GB
ebs volume             vol-0b1e736ba05cb7758
ebs volume type        gp3
provisioned IOPS       16000
provisioned throughput 1000 MB/s
io scheduler           none
nr_requests            63
read_ahead_kb          128

checkpoint_completion_target = 0.9
cluster_name = 18/main
data_directory = /var/lib/postgresql/18/main
effective_cache_size = 192 GB
effective_io_concurrency = 256
huge_pages = try
jit = off
maintenance_io_concurrency = 256
maintenance_work_mem = 102 GB
max_connections = 114
max_parallel_maintenance_workers = 16
max_parallel_workers = 32
max_wal_size = 32 GB
random_page_cost = 1.1
server_version = 18.6 (Ubuntu 18.6-1.pgdg26.04+2)
shared_buffers = 64 GB
wal_level = replica
work_mem = 1 GB