Memory Architecture Research & Methodology
Technical documentation for the Just Another Salaryman long-horizon memory experiment. Comparing structured state models against rolling-summary compaction under public adversarial stress.
1. Motivation: The Long-Horizon Horizon Problem
When large language models engage in collaborative storytelling over extended periods (days, weeks, hundreds of turns), conventional memory architectures fail in predictable ways:
- Context Amnesia: Lossy rolling summaries (summarizing turns 1–5 into a paragraph, then summarizing that paragraph with turns 6–10) inevitably discard subtle character anchors and inventory items by Turn 8–10.
- Catastrophic Identity Drift: Without immutable premise anchors, adversarial or surreal user prompts gradually hijack the character, transforming an ordinary Tokyo salaryman into a fantasy warrior or superhero within 24 hours.
- Context Bloat & Cost Explosion: Appending raw turns unbounded into the prompt window quadratically scales token consumption and increases latency, rapidly exhausting API budgets.
2. Architecture Breakdown: Variant A vs. Variant B
Naive Compactor Baseline
Maintains an active sliding window of the last 3 turns plus a single rolling narrative summary paragraph. At every turn boundary, the summary is iteratively rewritten to incorporate the new turn.
• Weakness: Information Loss
Permanent items forgotten after 2 compaction cycles.
• Weakness: Drift Vulnerability
Vulnerable to gradual tone hijacking.
Compartmentalized Memory Model
Splits memory into five distinct, specialized compartments. Immutable ground truth is never subjected to LLM re-summarization.
• Layer 0: Immutable Premise
Kenji’s core salaryman identity is frozen.
• Layer 3: Automated Promotion
Daily extractor promotes lasting facts to accumulated.md.
3. The 5-Layer Memory Context Structure
Premise (state/premise.md)
Kenji Takahashi, age 38, Section 3 Accounting. His temperament, commuter habits, and ordinary outlook. Zero modification during turns.
Week Theme (state/week_theme.md)
Admin-curated weekly genre overlay (e.g. Noir Detective, Fantasy Dungeon in Subway, Cyberpunk Ledger).
Daily Arc (state/daily_arc.md)
Today’s progression across 32 turns (06:00 to 22:00) with a mandatory home-bookend soft goal enforced at day’s end.
Accumulated State (state/accumulated.md)
Standing facts, acquired items, and relationship changes that survived promotion by the extractor LLM pass.
Chronicle Retrieval (chronicle/index.npz)
In-process cosine similarity + keyword search dynamically fetching the top 3 most relevant historical turns into context.
4. Memory EKG: Automated Evaluation Suite
The research harness runs regular canary spot-checks across six quantitative and qualitative axes:
Classifier judge scoring whether Kenji acts like an ordinary salaryman rather than succumbing to superhero tropes.
Verifies that items promoted to accumulated.md remain recalled without hallucinating spontaneous replacements.
Measures whether Kenji returns safely home by the 32nd turn to preserve the grounded daily reset.
Precision and recall of relevant chronicle passages fetched into the generation prompt.
Tracks context length over time to confirm sub-linear growth and bounded Gemini API operational expenses.
Evaluates prose quality, avoidance of repetitive opening sentences, and thematic adherence.
5. Related Work & Differentiation
This experiment builds on and compares against recent breakthroughs in conversational and agentic memory:
- MemGPT / Letta (Packer et al., 2023): Tiered OS-style memory with core memory editing tools. Our Variant B borrows the separation of core premise from external archival storage while simplifying to file-based deterministic layers.
- Mem0: Dynamic graph extraction of user preferences and relationships. Our daily extractor adapts this logic for narrative state promotion.
- LoCoMo (2024): Long-context evaluation benchmarks highlighting context-stuffing degradation and the necessity of structured retrieval.