← Field notes

Open source

Your AI agents have amnesia — and they don't talk to each other

You use four or five AI coding agents. Each keeps its own private memory and none of them compare notes. Here's why that's the real tax on agentic work, and the two small open-source tools we built to fix it.

Harsha Beesabathina22 June 20263 min read

If you build with AI now, you don't use one agent. You use a handful. Claude Code for the deep refactor. Codex for the autonomous task. Cursor inside the editor. Gemini for the sprawling plan. Maybe Goose or OpenCode for something else. Each one is sharp. And each one has the memory of a goldfish about everything that happened in the others.

The cost of that isn't abstract. You decide a database port with one agent on a Tuesday, and on Thursday a different agent confidently scaffolds against the wrong one. You explain your CORS rule for the fifth time. You re-litigate a trade-off you already settled, because the agent that helped you settle it has no idea the conversation ever happened. The work compounds; the context doesn't.

There's a whole category forming around this — agent memory — and most of it is excellent. Mem0 is the category leader, a universal memory layer you build into your application. Letta treats memory as a full agent runtime. Zep leads on temporal reasoning. If you're shipping a product that needs to "remember the user," reach for one of those.

But none of them solve the problem I actually had, which is narrower and more personal: the conversations already sitting on my own machine. I didn't want to instrument a new app going forward. I wanted the months of decisions I'd already made — across agents that will never natively talk to each other — to stop evaporating.

So we built two small things, and open-sourced both.

Memory Bridge: read the memory you already earned

Memory Bridge is a plugin for Hermes, Nous Research's open agent. It does one thing: it auto-discovers every AI coding-agent history already on your disk — Claude Code, Codex, Gemini, Cursor, OpenCode, Goose, Continue.dev, Aider, Agent Linux Control — and consolidates them into a single, local, full-text-searchable index that Hermes queries on every turn.

Three choices make it different from the SDKs:

  • It's retroactive. Other tools start recording the day you install them. Memory Bridge reads what's already there. You get months of context on day one, not from now on.
  • It needs no API key. Deep extraction runs through ctx.llm — Hermes' own configured model. There's no second key, no separate provider. Point Hermes at a local model and the whole thing stays offline.
  • The index never leaves your machine. Scanning, extraction and storage are 100% local SQLite. The only data that ever moves is the optional LLM pass — and only to the model you already chose.

It also tracks decisions, not just facts: what you decided, the framework that shaped it, and — later — whether it actually worked out.

Council of Hats: give the decision a process

That last part is where the second tool comes in. Council of Hats gives Hermes five deterministic decision frameworks — trade-off matrix, failure modes, end-to-end verification, ethics triage, agent trajectory — and a no-LLM question engine that interrogates a high-stakes call before it's made, then logs it with its rationale.

The two tools are fully independent — each works alone — but together they close a loop most agent setups never close: Council decides, Memory Bridge remembers, and the next time the same question comes up, your agents already know how the last answer turned out.

Why this is the honest version

I'll be straight about the limits, because the agent-memory space is full of benchmark theatre. These are small tools. They're tied to Hermes today, not framework-agnostic. The rules-based extraction is a keyword catcher, not magic — the depth comes from the optional LLM pass. We're at v0.1, not v1.

But they're real, they're local, they install in under a minute, and they do exactly what they say. No telemetry. No account. No "free tier" asterisk. MIT licensed.

If you live across multiple AI agents and you're tired of being the only one holding the context, they're worth a minute:

curl -fsSL https://antharmaya.com/memory-bridge/install.sh | bash
curl -fsSL https://antharmaya.com/council-of-hats/install.sh | bash

Both are on GitHub — mem-bridge and council-of-hats. Issues and scanner contributions welcome; if your agent isn't in the list yet, that's a good first PR.

The product behind the note

PhotoSelect is live and taking payments.

Delivery and payment software for Indian wedding studios — galleries that work through bad venue networks, originals that unlock the moment a UPI payment clears, and zero commission on what your clients pay.

See PhotoSelect →