← all runs

Cohere-1M — 2026-10-08

QPS at recall

Throughput under load

Each configuration is measured at its operating point for the target: its fastest sweep point whose recall reaches it. Values are measured there, not interpolated, so the recall can be above the target. Each cell is wall-clock QPS over the whole run, then in parentheses the throughput while every client is busy, which leaves out the drain at the end, and below them the p50 and p95 query latency over all clients. With more than one client, N× is the speedup: wall-clock QPS over the same configuration's single-client wall-clock QPS at that target. Ideal scaling matches the client count until the host's CPUs saturate, after which it flattens. Clients are concurrent connections from rekall, and CPU counts are the rekall host's. Wall-clock QPS includes rekall's own work between queries, so the 1-client row reads a little below QPS at recall, which comes from query latency. Recall is distance-based (ann-benchmarks, tie-aware).

Measured on config — mkt-rerank0 @0.95: mkt-rerank0 (r=0.953) · mkt-rerank160 @0.95: mkt-rerank160 (r=0.953) · mkt-rerank40 @0.95: mkt-rerank40 (r=0.952) · mkt-rerank640 @0.95: mkt-rerank640 (r=0.953) · mkt-rerank0 @0.99: mkt-rerank0 (r=0.994) · mkt-rerank160 @0.99: mkt-rerank160 (r=0.994) · mkt-rerank40 @0.99: mkt-rerank40 (r=0.992) · mkt-rerank640 @0.99: mkt-rerank640 (r=0.994)

Latency at recall

All values are taken at exactly the stated recall: each configuration's own sweep is log-interpolated between its two surrounding measured points (measured values are used verbatim on an exact hit) and the best configuration at that recall is shown — values are never interpolated across different configurations. “—” means no single sweep spans that recall; nothing is extrapolated. Recall is distance-based (ann-benchmarks, tie-aware). Insert/update/delete workloads are not measured.

Plots

Host

Versions

Search

Index builds

mkt-defaults: {"boundary_epsilon":null,"centroid_compression":null,"centroid_fastscan":null,"fan_out":null,"fastscan":null,"fastscan_bits":null,"nlist":null,"rerank_vectors":null,"soar_lambda":null}

instance type          c8g.medium
vCPU (observed)        1
memory (observed)      1.8 GiB
cpu model              unknown
kernel                 7.0.0-1014-aws
data directory         /var/lib/postgresql/18/main
mount point            /
filesystem             ext4
media                  ebs
device                 nvme0n1
device size            107 GB
ebs volume             vol-04865ecb7ba623cf5
ebs volume type        gp3
provisioned IOPS       3000
provisioned throughput 125 MB/s
io scheduler           mq-deadline
nr_requests            126
read_ahead_kb          128

checkpoint_completion_target = 0.9
cluster_name = 18/main
data_directory = /var/lib/postgresql/18/main
effective_cache_size = 1 GB
effective_io_concurrency = 200
huge_pages = try
jit = off
maintenance_io_concurrency = 200
maintenance_work_mem = 384 MB
max_connections = 52
max_parallel_maintenance_workers = 1
max_parallel_workers = 1
max_wal_size = 4 GB
random_page_cost = 1.1
server_version = 18.6 (Ubuntu 18.6-1.pgdg26.04+2)
shared_buffers = 512 MB
wal_level = replica
work_mem = 16 MB