Agent-memory scores 52.9% on LongMemEval-S, 17 points ahead of MemCore

Agent-memory is a long-term memory runtime for AI agents, published by tigerless-labs on 2026-09-01 and up to 605 stars and 35 forks on GitHub as of this morning. The project is Python, has no PyPI release yet, and installs from a checkout with uv. It was released in a month when several agent memory projects have appeared, and it makes its case with a paired benchmark against a named competitor.

What is in the store

The store is a directory of Markdown files. Each memory is one file with frontmatter that carries a stable name, a one-sentence abstract, a type, links, and provenance, and the body is free Markdown. Beside the files sits a SQLite index, which the README calls a rebuildable cache: the invariant, enforced by a test in the repository, is that deleting the index and running mem rebuild loses zero knowledge. The tree stays browsable with ls and grep, and portable off the system.

Retrieval answers with paths. A recall returns an L0 list of one-line abstracts and file paths, and the agent decides how deep to read each hit: outline, abstract, or full file. Three read tracks run in parallel so that any one of them can find a memory: a deterministic MEMORY.md injection at session start, BM25 recall over an FTS5 index (with a vector plugin fused in by RRF when enabled), and the plain directory tree reachable with ls and grep.

The number

The proof section measures agent-memory on LongMemEval-S with a bounded haystack of 12 sessions per episode, 120 episodes, using claude -p on Haiku 4.5 as the host and a calibrated Sonnet 5 as the judge, with two exam replays per arm. Pooled accuracy: agent-memory at 127 of 240, or 52.9%; MemCore at 86 of 240, or 35.8%; the no-memory arm at 7 of 120, or 5.8%. The paired difference against MemCore is +37 wins and 17 losses (p=0.009), and +35 and 14 (p=0.004) on the second replay.

The README is direct about what the numbers do not prove. The haystack is bounded, so the absolute figures cannot be compared to published LongMemEval scores; the study measures write strategy, and the system-to-system row differs in write and read together, so the gap belongs to the end-to-end system.

A second measurement covers hosts sharing the store: all nine ordered writer/reader pairs across Claude Code, Codex CLI, and Hermes pass, with a pooled net contribution over the no-memory arm of 2 of 36 to 13 of 36 (p=0.0074). What one host writes through its shell, another finds through its own.

Sleep-time consolidation, and no keys

The Manage layer runs on its own schedule, using the judgement of the host CLI so the library includes no LLM client and requires no API key. An unattended pass may add and update memories; deletion arrives only as a proposal the user confirms. Supersede leaves the chain intact, and a recall can be answered as of a past date. Writes happen at conversation boundaries, and the full session trace is archived first, so anything the distiller misses stays recoverable from the archive.

Setup is one command per host: mem setup --host claude-code or --host codex probes the host and appends a hook. The two open questions before adopting this on a real project are whether the LongMemEval gap survives on a longer haystack, and whether the Markdown-first design keeps working as a store outgrows a single laptop. The full measurement ledger and raw run records live in docs/experiments.md and experiments/, so the numbers can be re-run.

Source

The tigerless-labs/agent-memory repository on GitHub.