← all runs

Cohere-1M — 2026-10-07

QPS at recall

Throughput under load

Each configuration is measured at its operating point for the target: its fastest sweep point whose recall reaches it. Values are measured there, not interpolated, so the recall can be above the target. Each cell is wall-clock QPS over the whole run, then in parentheses the throughput while every client is busy, which leaves out the drain at the end, and below them the p50 and p95 query latency over all clients. Clients are concurrent connections from rekall, and CPU counts are the rekall host's. Wall-clock QPS includes rekall's own work between queries, so the 1-client row reads a little below QPS at recall, which comes from query latency. Recall is distance-based (ann-benchmarks, tie-aware).

Measured on config — PRISM @0.95: mkt-defaults (r=0.967) · pgvector HNSW (halfvec) @0.95: hnsw-m24-halfvec (r=0.955) · PRISM @0.99: mkt-defaults (r=0.991) · pgvector HNSW (halfvec) @0.99: hnsw-m24-halfvec (r=0.993)

Latency at recall

All values are taken at exactly the stated recall: each configuration's own sweep is log-interpolated between its two surrounding measured points (measured values are used verbatim on an exact hit) and the best configuration at that recall is shown — values are never interpolated across different configurations. “—” means no single sweep spans that recall; nothing is extrapolated. Recall is distance-based (ann-benchmarks, tie-aware). Insert/update/delete workloads are not measured.

Interpolated on config — PRISM @0.90: mkt-defaults · pgvector HNSW (halfvec) @0.90: hnsw-m24-halfvec · PRISM @0.95: mkt-defaults · pgvector HNSW (halfvec) @0.95: hnsw-m24-halfvec · PRISM @0.99: mkt-defaults · pgvector HNSW (halfvec) @0.99: hnsw-m24-halfvec

Build comparison

Build stats are for the configuration backing each column's best operating point: PRISM = mkt-defaults, prewarmed · pgvector HNSW (halfvec) = hnsw-m24-halfvec, prewarmed.

Plots

Host

Versions

Search

Index builds

hnsw-m24-halfvec: {"ef_construction":200,"encoding":"halfvec","m":24}
mkt-defaults: {"boundary_epsilon":null,"centroid_compression":null,"centroid_fastscan":null,"fan_out":null,"fastscan":null,"fastscan_bits":null,"nlist":null,"rerank_vectors":null,"soar_lambda":null}

instance type          c8i.xlarge
vCPU (observed)        4
memory (observed)      7.6 GiB
cpu model              Intel(R) Xeon(R) 6975P-C
kernel                 7.0.0-1014-aws
data directory         /var/lib/postgresql/18/main
mount point            /
filesystem             ext4
media                  ebs
device                 nvme0n1
device size            107 GB
ebs volume             vol-050db44c229f82c47
ebs volume type        gp3
provisioned IOPS       3000
provisioned throughput 125 MB/s
io scheduler           none
nr_requests            63
read_ahead_kb          128

checkpoint_completion_target = 0.9
cluster_name = 18/main
data_directory = /var/lib/postgresql/18/main
effective_cache_size = 6 GB
effective_io_concurrency = 200
huge_pages = try
jit = off
maintenance_io_concurrency = 200
maintenance_work_mem = 3 GB
max_connections = 58
max_parallel_maintenance_workers = 2
max_parallel_workers = 4
max_wal_size = 16 GB
random_page_cost = 1.1
server_version = 18.6 (Ubuntu 18.6-1.pgdg26.04+2)
shared_buffers = 2 GB
wal_level = replica
work_mem = 64 MB