Local Recall scaling benchmark

The tables combine standard performance metrics with scenario names from the LTM benchmark. The definitions below explain how to interpret them.

Patterns

The number of Patterns in the benchmark dataset. This is a count of LTM memory units, not a count of users, agents, tokens or queries. 1K means 1,000; 1M means 1,000,000.

Open (s)

The recorded time to open an existing LTM dataset, expressed in seconds. This is separate from Recall query latency. The stable-state results for 1M and 10M were measured after Posting Compaction.

Peak Managed

The reported peak managed memory during the benchmark run. Managed memory is memory managed by the .NET runtime; it does not represent all memory used by the process.

Peak Working Set

The reported peak amount of process memory resident in physical RAM during the benchmark run. It includes more than managed memory and is separate from the configured LTM memory budget.

P50 / P95 / P99

Query latency percentiles. P50 is the median; P95 and P99 describe the slower end of the measured latency distribution. Approximately 95% of measured queries finish at or below P95, and 99% at or below P99. Values are expressed in milliseconds, and lower values indicate shorter latency.

Hot / Context

Hot and Context are benchmark scenario names. Hot identifies the Hot query workload; Context identifies the context-based query workload. Hot does not itself mean that the query result is cached. For example, Hot Cold-first is the Hot scenario measured under the benchmark's cold-first conditions.

Cold-first / Warm

Cold-first and Warm distinguish cache conditions in the benchmark. Cold-first identifies the initial-query measurements; Warm identifies the subsequent workload measurements. Warm does not guarantee a cache hit. These labels alone do not establish that the operating system's file cache was cleared.

Physical Reads

The physical-read counter reported by the benchmark for the corresponding scenario. It is not automatically a per-query average or a count of disk-device operations. Compare these counters only when the measurement scope and workload are comparable.

Throughput (qps)

The number of measured queries completed per second during the scenario. Throughput depends on the workload and concurrency; it is not necessarily the reciprocal of P50 latency.

Cache Hit / Miss

The cache-hit and cache-miss counters reported for the scenario, shown as Hit / Miss. These counters describe the instrumented cache, not every cache in the system. The 100% result-cache hit rate applies to the reported HotWarm and ContextWarm measurements.

Posting Compaction

A maintenance operation that rewrites posting data into a more compact layout. The tables compare measurements before this operation with stable-state measurements after it. Its first-run cost is reported separately from subsequent stable-state Open times.

These measurements concern local Storage and Recall performance. They exclude LLM response generation and do not establish production service guarantees or the quality of an agent's reasoning. Exact query construction, cache preparation and counter boundaries should be read together with the benchmark implementation and methodology.