Rekall benchmark runs

72 runs across 7 datasets

Cohere-100M

DateCommentInstancePRISM vs competition
2026-10-15 · 08:45:25—r8i.4xlargevs vchordrq: +499% @0.80 · +601% @0.89 · +760% @0.98
2026-10-11 · 15:36:41feat/rerank-pool-cache-aware (8f4edc4), auto settings, clean run after PG restartr8i.8xlarge—
2026-10-10 · 19:23:08Testing rerank pool (main)r8i.8xlarge—
2026-10-10 · 08:20:35Testing rerank pool, take 2r8i.8xlarge—
2026-10-09 · 18:05:55Testing rerank poolr8i.8xlarge—
2026-09-28 · 22:19:15Testing for regressions before public releaser8i.8xlarge—
2026-09-24 · 19:42:11Testing 100M after pg_vectorsearch renamer8i.8xlarge—
2026-09-04 · 09:38:17—r8i.8xlarge—
2026-09-03 · 16:49:25Testing new rerank_pool formular8i.8xlarge—
2026-09-03 · 10:01:57—r8i.4xlarge—
2026-09-02 · 10:39:20Is HNSW build practical on 16xlarge?r8i.16xlargevs pgvector HNSW (halfvec): +120% @0.80 · +74% @0.89 · +318% @0.99
2026-09-02 · 10:27:57Meerkat 100M build on 4xlarger8i.4xlarge—
2026-07-17 · 07:33:04Graviton: 8xlarge downsized from 24xlarger8g.8xlargevs pgvector HNSW (halfvec): +38% @0.80 · +386% @0.89 · +1021% @0.99
2026-07-16 · 12:25:00100M Cohere on Gravitonr8g.24xlargevs pgvector HNSW (halfvec): +69% @0.80 · +54% @0.89 · +198% @0.99
2026-07-16 · 08:16:38resized: 100M dataset build on 24xlarge downsized to 8xlarger8i.8xlargevs pgvector HNSW (halfvec): +95% @0.80 · +493% @0.89 · +1646% @0.99
2026-07-15 · 16:23:46100M dataset build on 24xlarge downsized to 8xlarger8i.24xlargevs pgvector HNSW (halfvec): +105% @0.80 · +74% @0.89 · +229% @0.99
2026-07-13 · 18:20:39—r8i.24xlargevs pgvector HNSW (halfvec): +157% @0.80 · +91% @0.89 · +300% @0.99
2026-07-11 · 23:08:03—r8i.24xlargevs pgvector HNSW (halfvec): +116% @0.80 · +53% @0.89 · +305% @0.99
2026-07-11 · 19:11:10100M resized from 24xlarge to 4xlarger8i.4xlargevs pgvector HNSW (halfvec): +505% @0.81 · +16038% @0.89 · +38891% @0.98
2026-07-10 · 23:31:17Higher boundary_replicationr8i.24xlarge—
2026-07-10 · 12:32:55—r8i.24xlarge—
2026-07-07 · 14:43:42smallest-practical probe: 100M on 128GB/8 workersr8i.4xlarge—
2026-07-07 · 09:15:52—r8i.24xlargevs pgvector HNSW (bit): +795% @0.95 · +866% @0.95 · +942% @0.95
vs pgvector HNSW (halfvec): +128% @0.80 · +57% @0.89 · +202% @0.99

Cohere-10M

DateCommentInstancePRISM vs competition
2026-10-09 · 16:12:07Testing rerank poolr8g.4xlarge—
2026-10-09 · 15:44:23Testing auto rerank poolr8g.4xlargevs pgvector HNSW (halfvec): +178% @0.80 · +89% @0.90 · +271% @0.99
2026-07-16 · 09:21:5310m Cohere on Graviton 4xlarger8g.4xlargevs pgvector HNSW (halfvec): +121% @0.80 · +65% @0.90 · +291% @0.99
2026-07-12 · 16:59:48Exact centroid build branchr8i.4xlargevs pgvector HNSW (halfvec): +156% @0.80 · +144% @0.90 · +405% @0.99
2026-07-10 · 22:54:24Higher boundary_replicationr8i.4xlargevs pgvector HNSW (halfvec): +201% @0.80 · +123% @0.90 · +299% @0.99
2026-07-07 · 07:50:42—r8i.4xlarge—
2026-07-06 · 14:01:48—r7g.2xlarge—
2026-07-03 · 16:15:06—r8i.4xlargevs pgvector HNSW (halfvec): +53% @0.80 · +15% @0.90 · +109% @0.99
2026-07-03 · 16:08:44—r8i.4xlargevs pgvector HNSW (halfvec): +85% @0.80 · +49% @0.90 · +146% @0.99
2026-06-24 · 18:26:56—r8i.4xlargevs pgvector HNSW (halfvec): +98% @0.80 · +56% @0.90 · +159% @0.99
2026-06-16 · 10:18:46—r8i.8xlargevs vchordrq: +66% @0.80 · +109% @0.90 · +305% @0.99
vs vchordrq-rabitq4: +1173% @0.80 · +1049% @0.90 · +903% @0.99
vs vchordrq-rabitq8: +349% @0.80 · +363% @0.90 · +891% @0.99
2026-06-15 · 15:18:40—r8i.8xlarge—
2026-06-12 · 21:44:04—r8i.2xlargevs pgvector HNSW: +16% @0.80 · +65% @0.90 · +115% @0.99
vs pgvector HNSW (bit): +154% @0.80 · +285% @0.89 · +4219% @0.98
2026-06-12 · 15:25:10—r8i.16xlarge—
2026-06-11 · 22:43:29—r8i.16xlargevs pgvector HNSW: +18% @0.80 · +9% @0.90 · +75% @1.00
vs pgvector HNSW (bit): +118% @0.80 · +149% @0.89 · +314% @0.98

Cohere-1M

DateCommentInstancePRISM vs competition
2026-10-13 · 09:53:23Rerun of sweepc8g.large—
2026-10-12 · 21:28:34Testing rerank cache settingsc8g.large—
2026-10-12 · 11:47:151M with increased IOPSc8g.large—
2026-10-09 · 10:55:55recall-floor 1.2 coefficient verification (PR #252 @ 4e5b29b)c8g.medium—
2026-10-09 · 07:47:51recall-floor verification (pg_vectorsearch PR #252 @ 97a221a)c8g.medium—
2026-10-08 · 20:32:59cost-aware rerank_pool scale sweep (pg_vectorsearch PR #252, 938d068)c8g.medium—
2026-10-08 · 12:51:28cohere-1m c8g.medium query sweep phase 1: nprobe tail to 640, rerank pools 40/160/640/0c8g.medium—
2026-10-08 · 10:15:12c8g.medium rung, chunk_size=10000 via dataset config (OOM workaround)c8g.medium—
2026-10-07 · 20:59:58Testing multi-clientc8g.largevs pgvector HNSW (halfvec): +427% @0.88 · +608% @0.94 · +12656% @1.00
2026-10-07 · 14:53:32Testing concurrent clientsc8i.xlargevs pgvector HNSW (halfvec): +286% @0.88 · +90% @0.94 · +582% @1.00
2026-09-08 · 14:34:32Meerkat 1M on c8g.mediumc8g.medium—
2026-09-03 · 08:51:47Prewarm fix: warm the sweep's widest setting before any timed point (rekall@90330ec)c8g.xlargevs pgvector HNSW (halfvec): +141% @0.88 · +70% @0.94 · +453% @1.00
2026-09-02 · 12:28:30Try to match HNSW on c8.xlarge with Meerkat on c8.largec8g.xlargevs pgvector HNSW (halfvec): +40% @0.97 · +43% @0.98 · +470% @1.00
2026-08-29 · 07:35:32—c8g.largevs pgvector HNSW (halfvec): +4422% @0.82 · +3902% @0.91 · +18779% @1.00
2026-08-26 · 22:07:17cohere-1m ladder: c8g.large 4 GiBc8g.largevs pgvector HNSW (halfvec): +4676% @0.82 · +3591% @0.91 · +7137% @0.99
2026-08-26 · 19:21:39cohere-1m ladder: m8g.large 8 GiBm8g.largevs pgvector HNSW (halfvec): +120% @0.88 · +39% @0.94 · +69% @0.99
2026-08-26 · 12:43:33cohere-1m warm reference, builds the indexesm8g.largevs pgvector HNSW (halfvec): +112% @0.88 · +34% @0.94 · +88% @0.99
2026-08-11 · 14:33:04—r8i.24xlargevs pgvector-ivfflat: +3040% @0.87 · +4275% @0.94 · +5335% @1.00
vs pgvector-ivfflat (bit): +1626% @0.87 · +2049% @0.93 · +3923% @0.99
vs pgvector-ivfflat (halfvec): +2044% @0.87 · +2718% @0.93 · +7430% @1.00

Cohere-50M

DateCommentInstancePRISM vs competition
2026-07-13 · 11:30:38—r8i.2xlargevs pgvector HNSW (halfvec): +149% @0.80 · +143% @0.89 · +79698% @0.99
2026-07-12 · 17:03:48Exact centroid build branchr8i.12xlargevs pgvector HNSW (halfvec): +193% @0.80 · +127% @0.89 · +338% @0.99
2026-07-11 · 16:04:57—r8i.4xlargevs pgvector HNSW (halfvec): +119% @0.80 · +69% @0.89 · +10057% @0.99
2026-07-11 · 09:12:0950M resized from 12xlarge to 4xlarger8i.4xlargevs pgvector HNSW (halfvec): +110% @0.80 · +64% @0.89 · +10021% @0.99
2026-07-10 · 23:21:52Higher boundary_replicationr8i.12xlargevs pgvector HNSW (halfvec): +163% @0.80 · +100% @0.89 · +286% @0.99
2026-07-10 · 12:04:56—r8i.12xlargevs pgvector HNSW (halfvec): +123% @0.80 · +65% @0.89 · +271% @0.98
2026-07-09 · 22:25:00—r8i.12xlargevs pgvector HNSW (halfvec): +150% @0.80 · +67% @0.89 · +210% @0.98
2026-07-03 · 16:34:51—r8i.12xlargevs pgvector HNSW (bit): +488% @0.95 · +520% @0.95 · +553% @0.96
vs pgvector HNSW (halfvec): -2% @0.87 · +9% @0.93 · +127% @0.99
2026-07-03 · 16:30:21—r8i.12xlargevs pgvector HNSW (bit): +526% @0.95 · +552% @0.95 · +579% @0.96
vs pgvector HNSW (halfvec): +2% @0.88 · +14% @0.93 · +132% @0.98
2026-06-26 · 13:14:00—r8i.12xlargevs pgvector HNSW (bit): +481% @0.95 · +519% @0.95 · +559% @0.96
vs pgvector HNSW (halfvec): -1% @0.87 · +7% @0.93 · +116% @0.99
2026-06-23 · 13:26:42—r8i.16xlargevs pgvector HNSW: +9% @0.90 · +7% @0.94 · +33% @0.98
2026-06-23 · 13:06:47—r8i.16xlargevs pgvector HNSW: +17% @0.90 · +21% @0.94 · +54% @0.98
vs pgvector HNSW (halfvec): +27% @0.90 · +25% @0.94 · +67% @0.98
2026-06-13 · 10:00:20—r8i.16xlargevs pgvector HNSW: -6% @0.80 · -16% @0.89 · +82% @0.99
vs pgvector HNSW (bit): +179% @0.80 · +223% @0.88 · +357% @0.96

Cohere-5M

DateCommentInstancePRISM vs competition
2026-06-09 · 20:15:18—ip-172-31-4-132vs pgvector HNSW: +63% @0.90 · +149% @1.00

Glove-100

DateCommentInstancePRISM vs competition
2026-08-12 · 10:51:17—r6i.16xlargevs pgvector HNSW: +29% @0.80 · +51% @0.89 · +132% @0.98
vs pgvector HNSW (halfvec): +26% @0.80 · +45% @0.89 · +124% @0.98
vs pgvector-ivfflat: +383% @0.80 · +604% @0.90 · +931% @0.99
vs pgvector-ivfflat (halfvec): +355% @0.80 · +559% @0.90 · +947% @0.99

Laion-5M

DateCommentInstancePRISM vs competition
2026-08-13 · 15:38:07—i4i.xlargevs pgvector HNSW: +91% @0.81 · +101% @0.90 · -65% @0.99
vs vchordrq: +37% @0.80 · +47% @0.89 · -67% @0.99