Engineering Memory for AI Agents

This report synthesizes recent developments in memory systems for AI Agents, focusing on the architectural shifts, practical implementations, and engineering trade-offs observed in leading products.

Executive Summary

Over the past year, the design of AI Agent "memory" has evolved from simply cramming more history into the context window to a more engineered, multi-layered system. The current approach treats the active context window as working memory, stores conversation logs, screen traces, and other records as episodic memory, maintains stable preferences, knowledge, and rules as semantic memory, and externalizes skills and procedures into a layer analogous to procedural memory. Products from Anthropic, OpenAI, OpenClaw, Hermes, and Cursor, despite different interfaces, are all solving the same core problem: providing agents with a persistent, consistent, and auditable long-term state under constraints of limited context, acceptable latency, and controlled cost.

Architectural approaches can be broadly categorized into four types:

1. File-based Memory: Examples include Claude Code, Hermes, and OpenClaw. Long-term memory is stored in Markdown files or directories. The pros are auditability, editability, and easy migration; the cons are weak structure and limited retrieval granularity.
2. Session Store + Retrieval-based Memory: Examples are Hermes (SQLite+FTS5), OpenClaw (SQLite/QMD/LanceDB), and Cursor's code indexing. This offers scalability and on-demand recall but requires managing index freshness, concurrency, consistency, and cost.
3. Background Synthesis-based Memory: Examples include Codex Memories, OpenClaw Dreaming, and Hermes external provider sync. These systems externalize the "when to write, what to write, and when to merge" decisions from the main conversation.
4. Pre-retrieval Sub-agent Memory: Examples are OpenClaw Active Memory and Hermes provider prefetch. This turns "recall first, then answer" into a mandatory pre-step to reduce the chance of missing critical context.

From a product maturity and engineering reusability perspective, the most robust approach currently isn't a "unified, universal memory DB," but rather a two- or three-layer memory system: one small, stable layer always injected into the prompt; one large, cheap layer retrieved on-demand; and optionally, a background reflection/compilation layer responsible for distilling durable facts, preferences, and relationships from raw episodic data. This aligns with academic research: Generative Agents defined recall using relevance, recency, and importance; MemoryBank introduced forgetting curves; LongMem and MemGPT abstracted long-term memory into hierarchical memory banks; and Mem0 further operationalized the extract-merge-retrieve pipeline, reporting lower p95 latency and token costs on benchmarks like LOCOMO.

For practitioners, three key conclusions stand out:

1. Memory is not just a vector store. In coding agents, rules (AGENTS.md), skills, plans, session search, worktrees, and hook logs collectively constitute long-term state.
2. Memory updates must be treated as a lifecycle engineering problem, involving write thresholds, conflict merging, compression, forgetting, replay, backup, and auditing.
3. Security and traceability must be built-in. More automated memory layers require source annotation, injection scanning, scope isolation, user-visible review panels, and one-click forget capabilities.

Analytical Framework

This report breaks agent memory into four main dimensions:

• Working Memory: Active tokens in the current context window.
• Episodic Memory: Temporally organized conversations, operations, screens, logs, and journals.
• Semantic Memory: Cross-session stable preferences, rules, knowledge summaries, and entity relationships.
• Long-term Memory: The sum of all external states reusable in future sessions.

Notably, procedural memory—often implemented as rules, skills, hooks, commands, plan files, or wiki compilations rather than traditional database records—is critical in current products, though not always the primary focus.

Common representation formats include:

1. Plain Text Files: e.g., Claude Code's CLAUDE.md and auto memory, Hermes' MEMORY.md/USER.md, OpenClaw's MEMORY.md and memory/YYYY-MM-DD.md.
2. Embeddings & Indexed Chunks: e.g., Cursor's code chunk embeddings, OpenClaw's hybrid search, semantic retrieval via various providers.
3. Structured Objects or Derived Layers: e.g., OpenClaw memory-wiki's claim/evidence/provenance, Hermes Honcho's user model, Codex memories' layered management of summaries, durable entries, recent inputs, and supporting evidence.

A mature system often performs "assembly" rather than mere "retrieval." Anthropic explicitly views memory, compression, and tool clearing as different levers of context engineering. Hermes separates the cached, stable system prompt layer from API-call-time overlays. OpenClaw performs a silent memory flush before compaction. Cursor encourages agents to find context themselves using grep and semantic search. The true challenge isn't "having memory," but deciding what stays resident, what is pulled on-demand, what is merged in the background, and what must be discarded promptly.

Product Comparison

The table below compares five representative products within the same framework. Note: The examples are representative samples, not the entire market. Where implementation details aren't public, it's marked as "Not explicitly disclosed," indicating insufficient public information, not necessarily the absence of a capability.

Product	Public Memory Forms	Primary Representation	Storage Backend	Indexing & Retrieval	Update Strategy	Synthesis & Forgetting	Security & Privacy	Public Evidence
Claude Code	CLAUDE.md persistent instructions + auto memory; in-session compaction.	Markdown files. MEMORY.md as index; topic files loaded on-demand.	Local filesystem; one memory directory per repo, shared via worktree.	CLAUDE.md & auto memory entry loaded at startup; topic files read on-demand via file tools; no public vector index.	Auto memory enabled by default; Claude decides what's worth remembering; runtime read/write to memory files.	MEMORY.md limited to first 200 lines or 25KB; Anthropic uses compaction in long tasks, retaining key summaries and recently accessed files.	Memory is machine-local; autoMemoryDirectory not redirectable by project/local settings; Anthropic API memory tool supports client-side and ZDR.
Codex	Local cross-thread memories (Memories); Chronicle (screen context); AGENTS/skills as procedural memory.	Local Markdown memory files, split into summaries, durable entries, recent inputs, supporting evidence.	~/.codex/memories/ local directory; Chronicle extension stored in memories_extensions/chronicle.	Public docs don't detail ANN/DB specifics; can control use/generate memories; Chronicle uses screenshots, OCR, time info.	Generated only from "eligible prior threads"; skips active/short-lived threads; may skip near rate limits.	Background consolidation; supports thread-level toggles; memory summaries versioned and rebuilt when format becomes stale.	Generated fields undergo secrets redaction; can disable memory generation from "external context threads"; Chronicle local files are unencrypted and increase prompt injection risk.
Cursor	Publicly more like "persistent instructions + code indexing + plans/sub-agents"; no official mention of a dedicated auto-memory directory.	Rules/AGENTS.md/Skills as text; code index from syntactic chunks + embeddings.	Undisclosed specific index DB; public details: Merkle tree, simhash, chunk embeddings, index reuse & access proofs; Cloud Agents run in isolated VMs.	Agents use grep, semantic search, Explore sub-agent to pull context; .cursorignore & .cursorindexingignore control access and indexing scope.	Rules are long-lived; index updates asynchronously in background, reusing chunk cache embeddings; team-safe reuse of approximate codebase indexes.	/compress releases context space; Plan Mode converts long tasks to markdown plans; forgetting strategy for long-term "auto memory" not disclosed.	Index reuse via Merkle tree content proofs prevents unauthorized leakage; Hooks can observe/block agent loop; MCP requires source & permission auditing.
OpenClaw	MEMORY.md long-term memory + memory/YYYY-MM-DD.md diary layer + DREAMS.md; optional active memory, memory-wiki, QMD, LanceDB, Honcho.	Primarily Markdown, with derived layers like SQLite/LanceDB/QMD/Wiki claims.	Default workspace files + per-agent SQLite; optional QMD sidecar, LanceDB, local/external Honcho service.	Default memory search supports embeddings, keywords, or hybrid; Active Memory uses blocking sub-agent before main reply; QMD supports BM25+vector+rerank.	Memories written as plain Markdown; optional session transcript indexing enabled; dreaming promotes short-term signals to long-term memory in background.	Auto-compaction enabled by default; memory flush can run before compaction; Dreaming has light/deep/REM phases; session maintenance supports prune/cap.	Clearly distinguishes privacy from authorization; per-user session/memory isolation ≠ host permissions; workspace not a hard sandbox by default.
Hermes	Built-in MEMORY.md + USER.md + session_search + 8 external memory providers; also Honcho, Skills, Context Files.	Bounded text snapshots + SQLite session store + provider structured context.	~/.hermes/memories/ files; ~/.hermes/state.db SQLite WAL + FTS5; external providers can connect to cloud/self-hosted services.	At startup, bounded memory injected into system prompt; session_search uses FTS5; providers prefetch memories and inject context; Honcho does semantic search and user modeling.	Memory tool for add/replace/remove; providers sync after each turn & extract at session end; built-in memory writes also mirrored to provider.	Strict character limit; must consolidate/replace when exceeded; compression triggered at 50% pre-check and 85% gateway thresholds, flushing memory first, then protecting last N messages.	Memory entries & context files scanned for injection/exfiltration; SQLite WAL supports concurrent read/write; prompt caching pursues stable prefix.

From this table, we see:

• Claude Code & Codex lean towards "lightweight, visible local memory." Both emphasize user visibility, editability, and clearability for trust-building, but disclose fewer backend retrieval details. Claude Code is like "loading a condensed index always, then reading topic files on demand"; Codex is like "background extraction after a thread, then re-injecting durable/recent/evidence multi-layer files into subsequent threads."
• Cursor takes a different path. Its most detailed public disclosures are about code understanding indexes and context discovery, not "user preference memory." Its documentation emphasizes agents finding context via grep, semantic search, and Explore sub-agents, with backend use of Merkle trees for incremental change detection, syntactic chunking for embeddings, and simhash + content proofs for safe team index reuse. This is powerful for coding but means it doesn't yet offer an independent, auditable, lifecycle-defined product-grade memory system like Codex Memories or Claude Code auto memory for cross-session personal assistant scenarios.
• OpenClaw & Hermes represent a more "systematic," platform-oriented route. Both keep built-in file memory as a first layer for human auditability, migration, and repair, while introducing stronger retrieval layers and provider/plugin mechanisms. OpenClaw is more "layered" (raw diary, long-term memory, active memory sub-agent, dreaming consolidation, wiki compilation, various backends). Hermes emphasizes the "bounded stable prefix + session store + provider enhancement" combination, embedding prompt cache stability, WAL concurrency, FTS5, and context engine pluggability into its developer architecture docs. In terms of engineering reusability, these two most resemble buildable "memory substrates."

Memory Lifecycle & Context Engineering

A viable memory system shouldn't just consider "where it lives," but "how the entire lifecycle runs": collection, filtering, distillation, indexing, retrieval, injection, compression, expiration, and auditing. This workflow is explicit in mature products, though named differently (Anthropic's "context engineering," Codex's background extraction/consolidation, OpenClaw's Dreaming for promotion, Hermes' provider prefetch/assembly/compression/persistence as independent subsystems).

• Collection & Write Thresholds: Claude Code and Codex avoid "writing every turn." Claude lets the model judge what's worth remembering; Codex skips active/short-lived threads, generates memories only after a thread is idle for a while, and may abandon extraction if rate limit headroom is low. The essence: memory writing has cost, and premature writing can solidify unstable intermediate conclusions.
• Short-term to Long-term Synthesis: OpenClaw is a prime example, clearly separating working layers (memory/YYYY-MM-DD.md) from long-term layers (MEMORY.md), with Dreaming's light/deep/REM phases deducing, scoring, and promoting recent signals to durable memory (only allowing grounded snippets). Hermes is similar but more conservative, directly capping long-term memory at ~1300 tokens, forcing consolidation or replacement rather than unlimited growth.
• Recall Strategy: Best practice isn't "inject all long-term memory upfront." Anthropic advocates combining compaction, tool clearing, and memory. In Claude Code's long tasks, it compresses context, discards redundant tool outputs, and uses structured notes for cross-window bridging. OpenClaw's Active Memory goes further, making recall a blocking memory sub-agent before the main reply. Hermes' providers do prefetch per turn, but built-in memory stays a frozen snapshot. Codex Chronicle uses screen context to help agents find the source documents truly worth reading directly.
• Context Window Management: Products now prioritize "preserving cache, speed, and consistency." Hermes explicitly separates cached system prompt from API-call-time overlays, using 1-hour prompt caching; its built-in memory is a frozen snapshot from session start, written to disk but not altering the current prompt to protect the prefix cache. Anthropic's context engineering cookbook similarly emphasizes the compaction + tool clearing combo to avoid context rot. OpenClaw prunes old tool results in-memory without rewriting the disk transcript, balancing replay and cost.

Reusable Design Patterns

The following table extracts directly reusable engineering patterns from these products. For most teams, these patterns are more critical than "which vector database to choose."

Pattern	Problem Solved	Pros	Cons & Failure Modes	Best Suited For	Inspired By
Resident Index + On-Demand Details	Want to remember key facts across sessions without keeping many tokens resident.	Keeps startup context small and stable; good for prompt cache.	Agent must know when to dig into detail files; poor entry index can cause recall failure.	Coding, research, ops.	Claude Code's MEMORY.md + topic files; OpenClaw MEMORY.md + daily notes; Hermes bounded memory + session search.
Episodic vs. Semantic Layer Separation	Long history, but few reusable facts.	Facilitates merging, compression, forgetting, and provenance tracking.	Needs robust extraction criteria; risks writing transient state as long-term facts.	Long-term collaboration, multi-turn assistants, personal agents.	Codex summary/durable/evidence layers; OpenClaw working/long-term layers; Hermes MEMORY/USER vs. session DB.
Pre-Reply Recall Agent	Main agent often "forgets what it should remember."	Makes recall a mandatory pre-step, reducing omissions.	Adds visible latency; higher risk of prompt pollution; requires precise tool allowlists.	Non-coding personal assistants, customer service, long-task collaboration.	OpenClaw Active Memory; Hermes provider prefetch.
Background Synthesis & Idle Updates	Avoids writing unstable conclusions into long-term memory.	Reduces pollution; shifts extra token/reasoning costs to background.	Delay in visibility of new memory; background task failures can cause memory lag.	Desktop/local assistants, personal coding workflows.	Codex background memory generation; OpenClaw Dreaming; Hermes provider sync/extract.
Hybrid Search (Not Pure Vector)	Code, logs, paths, commands, short strings are not vector-friendly.	More robust for keywords, directories, symbol names, and semantics.	Requires weight tuning, deduplication, reranking, and metadata filtering.	Codebases, wikis, tickets, ops runbooks.	OpenClaw hybrid BM25+vector; Cursor grep+semantic search; Hermes FTS5 + provider semantic search.
Frozen Snapshots to Protect Cache	Frequent system prefix changes in long sessions cause cost/latency spikes.	Boosts prompt cache hit rate and stability.	Newly written memories in current session can't immediately reflect in prompt prefix.	High-frequency multi-turn sessions, Claude-family models.	Hermes frozen snapshot + prompt caching; Anthropic context engineering; Claude Code startup injection.
Source Provenance & Auditable Compilation	More automated memory risks "misremembering" and injection pollution.	Improves grounding, explainability, and delete/repair capabilities.	Complex implementation; needs claim/evidence schema and review UI.	Enterprise agents, compliance/knowledge workflows.	OpenClaw memory-wiki; Cursor content proofs; Codex supporting evidence; Hermes security scanning.
Pluggable Memory Provider	One memory system can't simultaneously satisfy personalization, graph, search, cost, and compliance needs.	Allows underlying memory to be swapped without changing main agent logic.	Provider semantic inconsistency; must handle migration, rollback, and observability.	Platform agents, B2B/SaaS, self-hosted.	Hermes memory providers; OpenClaw memory slots & multiple backends; Anthropic client-side memory primitive.

The single most important practical advice is: start with a two-layer memory system, then add a reflection layer, and only then consider complex graph layers. Separating a "small index always loaded" from a "large corpus searched on demand" solves most real-world problems. Only when facing cross-entity relationships, multi-agent identity isolation, or strong provenance requirements should you introduce Honcho, Wiki, Graph memory, or graph-backed providers. Academic work from Generative Agents (relevance/recency/importance), MemoryBank (forgetting curves), MemGPT/LongMem (hierarchical memory), and Mem0 (production-grade extract/merge/retrieve) broadly supports this engineering evolution from simple to complex.

Evaluation, Security & Engineering Constraints

Evaluation: Many teams still equate memory with "can we retrieve a snippet," but that's insufficient. Better metrics should include at least five categories:

1. Recall Quality: hit@k, evidence precision, grounding consistency.
2. Behavioral Benefit: task completion rate, cross-session consistency, reduction in repeated questions.
3. Efficiency: p50/p95 latency, token consumption, index rebuild time.
4. Lifecycle Quality: false remember rate, staleness, conflict resolution success rate, deletion propagation delay.
5. Operability: recovery capability, backup/restore time, observability coverage.

Mem0's work on LOCOMO highlights how memory architecture impacts not just answer accuracy, but also p95 latency and token costs. New benchmarks like LifeBench and AgentLongBench remind us that traditional static recall tests cannot fully measure agent memory in long-horizon, multi-source, interactive environments.

Security & Privacy: Current products split into two camps. File-based memory is easier for users to understand and delete, but risks local unencrypted plaintext (Codex Chronicle explicitly warns of unencrypted memory files increasing prompt injection risk; Claude Code auto memory is machine-local to reduce shared leakage). OpenClaw and Hermes emphasize scanning and isolation: OpenClaw states per-user isolation ≠ host authorization; Hermes scans memory entries and context files for injection/exfiltration. For enterprise products, the core isn't just "encryption yes/no," but scope, proof, auditing, minimal privilege, deletion capability, and thresholds for external context entering memory.

Scalability, Concurrency, Versioning: Hermes and Cursor provide the most detailed public engineering insights. Hermes uses SQLite WAL for multi-platform gateway multi-read/single-write, FTS5 queries, session lineage, and post-compression parent-child session relationships. Cursor uses Merkle trees, simhash, content proofs, and chunk embedding caches at the index layer, reducing time-to-first-query for similar team codebases from hours to seconds. Codex's changelog mentions versioning and stale-format rebuilding of memory summaries. These details show that memory systems must be operated like databases and distributed index systems, not just managed via prompt techniques.

Developer APIs & Integration: Maturity varies widely. Anthropic exposes a client-side memory tool as an API primitive. OpenAI provides Codex SDKs, App Server, MCP Server, and local memory controls. Cursor's public interfaces focus more on rules/Skills/Hooks/MCP/Cloud Agents webhooks. OpenClaw and Hermes make memory a replaceable slot/provider/plugin, offering more complete CLI and status interfaces. For platform teams needing composable capabilities, this presents a clear choice: If you're "building a product," pick systems with clear memory primitives and API surfaces. If you're "just boosting efficiency," file-based + rule-based approaches may already suffice.

Observability & Debugging: A clear trend is emerging: memory cannot be a black box. Claude Code offers /memory browsing. Codex provides /memories and inspectable local directories. OpenClaw has memory status, memory index --verbose, wiki status/doctor, and session cleanup/inspection. Hermes has a session DB, Web Dashboard, FTS5 search, tool call display, and plugin/provider status. Cursor at least exposes MCP logs, hooks, and Cloud Agent webhooks. If a team cannot answer "where did this memory come from, why was it recalled, how long until deletion takes effect, how fresh is the index?", its memory layer is likely not production-ready.

Limitations & Primary Sources

A key limitation of this report is that not all products have disclosed their underlying memory data structures, database choices, ANN algorithms, metadata schemas, conflict merging strategies, and deletion propagation mechanisms. Especially for Cursor, Claude Code, and Codex, while their user-facing memory experiences are well-documented, details on their underlying retrieval engines and sorting/merging logic are relatively scarce. Therefore, wherever official documentation is unclear, this report has noted it as "not explicitly disclosed."