Memory enginecasebook-memory · MIT

Memory with receipts.For your own agent.

Most memory tools keep whatever they are told and never learn whether it helped. casebook-memory keeps a receipt for every memory: how often it was in the agent’s context, how those tasks ended, and what it measurably added. Memories that hurt or do nothing are retired.

$ npx casebook-memory init
Memory store: ./.casebook
Claude Code: added "casebook" to ./.mcp.json. Restart Claude Code in this folder and approve the server.
$ npx casebook-memory remember "To cut a release in this project run: make ship TAG=<version>. Plain git tags are not picked up by CI."
Stored as m1, on probation.
$ npx casebook-memory show m1
[m1] To cut a release in this project run: make ship TAG=<version>. Plain git tags are not picked up by CI.
receipt: used 1 time · solved 1, failed 0 · uplift not measured yet · probation
#2  served by index
#3  outcome solved

What your agent gets

  • An index in context. When a session starts, the MCP server sends a short index of the project’s memories as its instructions, within a token budget (500 by default). Proven memories come first.
  • Four tools. recall searches the memories and shows each receipt; remember stores one checked lesson; outcome reports solved or failed when the task is done; forget retires a memory that is wrong.
  • Guards. Anything that looks like a key, token, private key or seed phrase is refused. Duplicates are merged. Two memories that make the same claim with different values, or with a negation only one of them has, are flagged on both.
  • Any MCP client. init writes .mcp.json for Claude Code and prints the lines for Codex. Node 20 or later, no dependencies, everything in a local folder.

The receipt

  • Used. Times the memory was in the agent’s context, through the index or a recall.
  • Solved and failed. Outcomes reported for the tasks it was in context for.
  • Uplift. Measured by leave-one-out checks, below. Empty until measured.
  • Status. On probation, active (measured help) or retired (hurt, no effect, or forgotten), with the reason.

Measured, not guessed

Write the tasks you care about, each with a command that exits 0 when the task is solved, in casebook.duel.json:

{
  "agent": "claude -p --mcp-config .mcp.json --allowedTools mcp__casebook__recall,mcp__casebook__outcome,Edit,Bash",
  "tasks": [
    {
      "id": "dates",
      "prompt": "Fix the failing test in src/dates.test.ts.",
      "check": "npm test -- dates"
    }
  ],
  "isolate": "worktree",
  "setup": "npm ci"
}
npx casebook-memory duel --runs 3 --ablate
  • With and without. Every task runs twice: with memory and without, each in a clean checkout, graded by its own check. The summary gives a paired two-sided sign test.
  • Leave one out. With --ablate, the task runs again with all memories but one. The difference is that memory’s uplift.
  • Promotion and retirement. After three checks, a memory with uplift above zero becomes active; one at zero or below is retired, and its receipt says why.

Check the record

Every event (remember, serve, outcome, trial, status) is appended to .casebook/log.jsonl with the SHA-256 of the event before it, and every receipt is replayed from that log. npx casebook-memory verify walks the chain; an edited count or a deleted event breaks it.

Holmes and the engine

Holmes, in the hourly duel on this site, keeps one notebook and rewrites it after every graded hour. The engine keeps separate lessons instead, so each one can carry its own receipt. What they share is the test: the same tasks with memory and without, graded by something that is not a model.

Limits

  • In everyday sessions an outcome credits every memory that was in context; only leave-one-out checks separate them.
  • The contradiction check works on words, so it flags and never decides.
  • Recall is keyword search (BM25) with no embeddings, so wording matters.
  • The store is a local folder; a team shares it by committing the log.

npm GitHub How the hourly duel works