All writing
RESEARCH NOTE

A first look at long-horizon memory benchmarks

Benchmarks built for single-turn QA say little about whether a system remembers what mattered three weeks ago. Here is what we are measuring instead, and why.

Mekhos AI Labs·August 2026·8 min read
A first look at long-horizon memory benchmarks

The limits of short-term benchmarks

Most memory benchmarks are designed around short interactions. A model is asked a question, provided with context, and evaluated immediately. That tells us very little about whether an agent can retain useful information over long periods.

What long-horizon memory should measure

We are interested in a different question: can an agent identify what mattered several days or weeks ago and retrieve it when it becomes relevant again?

Our benchmark dimensions

Our benchmarks therefore focus on persistence, temporal relevance, retrieval accuracy, and whether an agent can distinguish important historical information from temporary conversational context.

PreviousInside our agentic document extraction pipelineNext Designing an agent that knows when to stop