Why This Exists
A brief intro to why this came about.
Where it started
I have always had an interest in D&D and collaborative storytelling, but being as introverted as I am made it exceedingly hard to get started. Hard to find a group, none of my friends were interested, hard to find time to commit with work and family. So when AI started getting popular, I realized it was the perfect way to run my own private campaigns using an LLM as a Dungeon Master (DM). I’ve been testing, refining, and running several private one-shot campaigns to get a feel for what the models can do. It’s been great, not only as a game, but also as a hands-on learning opportunity for working with AI architecture.
After running one-shots for a while, I tried running a longer, multi-session campaign with no clear target or end goal. That’s when I started encountering story drift. To be honest, this project was born out of my frustration with having to constantly tell the LLM to fix plot points and story beats. These weren’t dramatic differences (I had spent a lot of time testing different memory structures and that seemed to help), but rather minor inconsistencies. Character A would refer to Character B by their real name when they weren’t supposed to know it yet, or the DM would forget items in my inventory until I reminded it. I spent a decent amount of prompts and effort between turns trying to get the narrative back on course and correcting details.
The part I didn’t notice for a while
I realized that because I was constantly correcting these details, I had no way to evaluate what the model was actually doing on its own. I couldn’t tell if the memory architecture was genuinely working, because I was acting as the de facto error-correction layer.
The idea behind this site is to experiment and see how the narrative unfolds without my intervention. The AI selected a random character name, Ryan Fujimoto, and updates to his day run every thirty minutes, whether or not I’m watching. The genre is randomly selected from a list each week. Any user can prompt and vote on his next action or upcoming events. With no one around to guide the story back on course if it wanders or restate a fact if it gets lost. If the memory holds, it holds on its own; if it drifts, it drifts in public where it can be measured.
More information about what is being compared, how drifts are scored, and the overall setup can all be found on the research page.
What I think I’ll be wrong about
Before running this, my guess was that a structured state would clearly beat a rolling summary. Most research disagrees for many tasks, suggesting that keeping the full context works just as well. I wrote my predictions down beforehand so I can’t change them. There’s a decent chance the final result shows it doesn’t matter much, but that still counts as a valuable result.