← all runs

Cohere-1M — 2026-10-07

QPS at recall

Throughput under load

Each configuration is measured at its operating point for the target: its fastest sweep point whose recall reaches it. Values are measured there, not interpolated, so the recall can be above the target. Each cell is wall-clock QPS over the whole run, then in parentheses the throughput while every client is busy, which leaves out the drain at the end, and below them the p50 and p95 query latency over all clients. With more than one client, N× is the speedup: wall-clock QPS over the same configuration's single-client wall-clock QPS at that target. Ideal scaling matches the client count until the host's CPUs saturate, after which it flattens. Clients are concurrent connections from rekall, and CPU counts are the rekall host's. Wall-clock QPS includes rekall's own work between queries, so the 1-client row reads a little below QPS at recall, which comes from query latency. Recall is distance-based (ann-benchmarks, tie-aware).

Measured on config — PRISM @0.95: mkt-defaults (r=0.968) · pgvector HNSW (halfvec) @0.95: hnsw-m24-halfvec (r=0.954) · PRISM @0.99: mkt-defaults (r=0.997) · pgvector HNSW (halfvec) @0.99: hnsw-m24-halfvec (r=0.993)

Latency at recall

All values are taken at exactly the stated recall: each configuration's own sweep is log-interpolated between its two surrounding measured points (measured values are used verbatim on an exact hit) and the best configuration at that recall is shown — values are never interpolated across different configurations. “—” means no single sweep spans that recall; nothing is extrapolated. Recall is distance-based (ann-benchmarks, tie-aware). Insert/update/delete workloads are not measured.

Interpolated on config — PRISM @0.90: mkt-defaults · pgvector HNSW (halfvec) @0.90: hnsw-m24-halfvec · PRISM @0.95: mkt-defaults · pgvector HNSW (halfvec) @0.95: hnsw-m24-halfvec · PRISM @0.99: mkt-defaults · pgvector HNSW (halfvec) @0.99: hnsw-m24-halfvec

Build comparison

Build stats are for the configuration backing each column's best operating point: PRISM = mkt-defaults, prewarmed · pgvector HNSW (halfvec) = hnsw-m24-halfvec, prewarmed.

Plots

Host

Versions

Search

Index builds

hnsw-m24-halfvec: {"ef_construction":200,"encoding":"halfvec","m":24}
mkt-defaults: {"boundary_epsilon":null,"centroid_compression":null,"centroid_fastscan":null,"fan_out":null,"fastscan":null,"fastscan_bits":null,"nlist":null,"rerank_vectors":null,"soar_lambda":null}

instance type          c8g.large
vCPU (observed)        2
memory (observed)      3.7 GiB
cpu model              unknown
kernel                 7.0.0-1014-aws
data directory         /var/lib/postgresql/18/main
mount point            /
filesystem             ext4
media                  ebs
device                 nvme0n1
device size            107 GB
ebs volume             vol-006ea9c5a53c4edff
ebs volume type        gp3
provisioned IOPS       3000
provisioned throughput 125 MB/s
io scheduler           none
nr_requests            63
read_ahead_kb          128

checkpoint_completion_target = 0.9
cluster_name = 18/main
data_directory = /var/lib/postgresql/18/main
effective_cache_size = 3 GB
effective_io_concurrency = 200
huge_pages = try
jit = off
maintenance_io_concurrency = 200
maintenance_work_mem = 1 GB
max_connections = 54
max_parallel_maintenance_workers = 1
max_parallel_workers = 2
max_wal_size = 16 GB
random_page_cost = 1.1
server_version = 18.6 (Ubuntu 18.6-1.pgdg26.04+2)
shared_buffers = 1 GB
wal_level = replica
work_mem = 32 MB