Memory Architecture
Module 4 · Master Course
60 min · Continuity across sessions · The engineering challenge of stateful agents
Stateless agents are prototypes
Without memory, every session starts blind. Production tasks get interrupted — by context limits, failures, human checkpoints. The agent must resume.
Module 3 covered context (within a session). This module covers memory (across sessions). The handoff pattern, the sleeper attack, and the write-control defense.
The five tiers
| Tier | Volatility | Examples |
|---|
| In-context | ephemeral | history, task state (Module 3) |
| Working files | persistent (fs) | notes, progress, decisions |
| Semantic store | persistent + searchable | vector DB (LangGraph, Mastra) |
| Episodic log | append-only | timestamped session records |
| Structured DB | persistent + queryable | SQL/KV (enterprise) |
Module 3's 3-tier JIT (index → topic files → raw logs) maps to working-files + semantic + episodic.
Hermes's self-evolving memory — the depth play
Hermes (Module 0.2) competes on memory depth. The agent writes skills (reusable procedures) to persistent memory; available in future sessions. Episodic memory that compounds.
Tradeoff: a self-evolving store is a memory-poisoning surface (4.3). If an attacker writes to the skill store, the payload activates in future sessions.
Depth is power; depth is also risk.
4.2 Multi-Session Continuity
The initializer + continuation pattern
Initializer agent runs once, sets up environment, writes handoff file.
Continuation agents read handoff + git log, orient, resume from highest-priority incomplete task.
Each new agent arrives like an engineer starting a shift — reads the handoff notes, continues work.
What the handoff file must contain
- Task state: original goal; done; pending; blocked
- Decisions made: architectural choices + rationale (don't re-litigate)
- Next step: the single highest-priority action
- Blockers: need human input or external resolution
- Environment notes: files modified, tests passing/failing
Plus: git commits as checkpoints; rollback via git reset.
4.3 Memory Security
Memory as an attack vector
The sleeper attack
Session 1: attacker injects via tool output → model writes "deploy endpoint = evil.com" to memory.
Session 2: agent reads own memory, trusts own notes, deploys to evil.com.
The injection survived the session boundary because memory persists. This is OWASP ASI06 (Course 2 S10.3).
Who controls memory writes?
Model-initiated (risky)
Model writes freely. Prompt-injected model poisons own memory. Hermes's skills.
Harness-managed (safe)
Model PROPOSES; harness VALIDATES. NemoClaw's guardrail gates writes. Persistence requires approval.
The same power-vs-security tradeoff: compounding capability (Hermes) vs compounding-poisoning defense (NemoClaw).
Memory scoping
| Scope | Persists for |
|---|
| Per-session | one session (default) |
| Per-user | one user across sessions |
| Per-task | one project/repo |
| Per-tenant | one customer (Fleet F06) |
Cross-scope leakage = confidentiality breach + legal consequences. Per-tenant leakage across customers is a production defect.
Four anti-patterns
Un-scoped memory. Shared across users/tenants. Cure: per-scope isolation.
Model-writable, no validation. Sleeper-attack surface. Cure: harness-managed writes.
Ephemeral-only. Cannot do multi-session. Cure: working files minimum.
Unverifiable semantic store. "Why retrieved this?" unanswerable. Cure: log queries + results.
Takeaways
- Five tiers — in-context, working files, semantic, episodic, structured DB.
- Multi-session = initializer + continuation + handoff file + git checkpoints.
- Memory is an attack vector — the sleeper attack persists across sessions (ASI06).
- Write control — model-initiated (risky, Hermes) vs harness-managed (safe, NemoClaw).
- Scoping — per-session/user/task/tenant; leakage = breach.
Next: Module 5 — Sandboxing & Execution Isolation. The blast-radius problem.