04. Memory

Getting It to Actually Know You Better Over Time: Memory Isn't One Big Pot, It's Three Drawers

Post 4, and personally, the one I think captures "the soul of a vertical product" best. So far we've made "this one writing task's" history survive a shutdown and resume — but that's still memory scoped to a single piece. SmartWriter's whole pitch is "gets better the more you write," and that runs on a different mechanism entirely: long-term memory that spans writing session after writing session and remembers you, the person. This post takes that slogan apart to see what it actually takes to build, and why I think a product has no soul at all if the details here aren't right.

🪧 "Gets Better the More You Write" Is Genuinely Hard to Pull Off

I wanted to say this slogan out loud from very early on: today's models are smart, but the text they produce still has that unmistakable "AI voice" clinging to it, and I wanted to build an agent that actually got my writing style. The moment I sat down to build it, though, I got stuck on a set of deceptively simple questions: what does "getting" someone even mean? What exactly should it remember, where does that live, who's allowed to write to it, and how do you keep it from learning the wrong lessons — while still letting it keep improving?

The naive approach: who cares what kind of memory it is, it's all "information about the user," so just dump it all into one place. Today's models all advertise million-token context windows, so "just put it in there" shouldn't be hard, right?

That doesn't work. Here's a concrete example: say a user is writing a piece venting about their job, and for emotional effect, drops a line like "I genuinely can't stand endless meetings." That's a throwaway line, specific to this one article. But if the memory system doesn't distinguish between layers, it might record that as "a piece of long-term information about this user." Next time they write a completely unrelated technical essay, the model might still be carrying "this person hates meetings" around, and keep looking for an excuse to work it in. That's a real failure — it recorded a one-off fact as a long-term preference, and even quietly started treating it as a "style" to pander to.

The hard part of "gets better the more you write" isn't "failing to remember" — it's misfiling: recording something that should've been forgotten as permanent, treating one article's context as if it applies to all of them, mistaking a single sentence's fact for a person's entire style. Put plainly: throw all your memory into one pot, and misfiling is inevitable. The fix isn't novel — it's a principle straight out of classic software engineering that's held up for decades: divide and conquer.

🧭 The Core Idea: Why Split Memory into Drawers?

To split it into drawers, you first need to know what axis to split along. In my view, what a writing assistant needs to remember boils down to three fundamentally different kinds of things:

• 🎨 Style: how they write. The words they reach for, sentence length, tone, what they'd never say. This is "voice" — the most subtle layer, the one closest to "soul."
• 📏 Rules: what they must do. Writing conventions and formatting requirements that have already been settled. This is the "hard rules" layer — spelled out, unambiguous.
• 📇 Facts: who they are, and what they've said. Background, a specific correction they made once, a temporary collaboration arrangement. This is "the file" — and it can update at any moment.

These three are fundamentally different in nature, and cramming them into one drawer guarantees conflicts.

Style demands slow, stable, resistant to contamination. A person's writing voice settles over a long stretch of time — you can't rewrite their profile just because one article happened to use an unusual word. The style layer is most vulnerable to one-off noise, and even a single contamination event makes the user feel like "wait, this thing understands me less than it used to."

Facts are the exact opposite — they demand fast, immediate, overwritable. They say today, "we're renaming this project to X," and that has to be recorded right away, and be fully overwritable by whatever they say tomorrow. Manage facts with the same "slow, stable" logic you'd use for style, and they're perpetually out of date.

Rules sit in between — stable, though not in the sense of slowly accruing like style, and not volatile like facts either. They're just a clear, explicit list of writing conventions.

With the "three drawers" framework sorted, let's look at what Claude Code CLI already gives you out of the box, and then what I had to build myself to cover the gap.

🔩 How Claude Code Does It: Two Drawers Already Exist — Missing Exactly One, "Style"

Good news: Claude Code never went down the one-pot road in the first place — it naturally has two drawers already. The ninth post in the Learn Claude Code source-walkthrough series covers Claude Code's memory mechanism in detail; I'll build on what it found.

One drawer is CLAUDE.md, which holds exactly "📏 Rules." It's a project-level standing-instructions file — write "what this project must always follow" into it, and it counts, every single time. The last two posts have already brought it up repeatedly.

The other is Auto Memory, which holds exactly "📇 Facts." It's a built-in layer of factual memory in Claude Code, where the model itself decides whether "this piece of information is worth remembering," and quietly accumulates facts like who you are, anything important you've said, agreements you've laid out before — automatically carrying them forward next time.

Both drawers a coding scenario actually needs are already there in Claude Code: rules (CLAUDE.md) and a file (Auto Memory). It's only missing the third one — "🎨 Style." That's not a knock against Claude Code — coding, by nature, doesn't demand much in the way of "personal voice." Whether code is correct and clear has objective standards; there's not much room for "this code needs to sound like me."

And that's exactly the layer writing can't afford to be missing. A writing agent exists to serve the user, and it only does that well if it "sounds like them." So the key gap I needed to fill for my writing agent was: build the "style drawer" Claude Code doesn't ship with.

🛠 What the SDK Gives You: Keys to Two Drawers, and Their Respective User Manuals

Before building anything, I needed to understand the "keys" to the two drawers the SDK already ships — specifically, how each one actually gets into the model's head, because that shapes where I could plug my own style layer in.

CLAUDE.md

An easy-to-get-wrong but important fact: it doesn't go into the system prompt. I originally assumed standing rules lived in that special system-prompt slot. They don't.

The SDK injects CLAUDE.md's content into "your first message" (wrapped in a <system-reminder> tag). The CLAUDE.md file on disk only gets read once, the moment the CLI subprocess starts — after that, it's cached in memory, and compaction doesn't trigger a re-read from disk. What actually makes it "resist compaction" is that the CLI re-injects a fresh <system-reminder>, built from that cached in-memory content, every single time it constructs a new user message. Compaction can sweep away the old injection sitting in conversation history, but it can't touch the new one that shows up in the next message (this mechanism was covered in post 2) — which is why it's remembered continuously.

One detail worth flagging: if the profile gets quietly changed by the backend mid-session, compaction alone doesn't make that session read the new content. It has to wait for that session's CLI subprocess to fully restart (say, if you close the app and reopen it much later) before it picks up the latest version on disk.

Auto Memory

A few details worth keeping in mind: it's on by default; it derives a storage path per code repository (repo), so subdirectories under the same repo share the same Auto Memory; and you can turn it off with one switch if you don't want it. The most important thing is that what it writes and what it doesn't is a call the model makes on its own — which is both its convenience (I don't have to manage it) and its risk (it might take it upon itself to record things I never wanted recorded, like blending in a style judgment).

With the pathways for rules and facts mapped out, the style drawer was still on me to route into the model's context — which is exactly what SmartWriter builds next.

🧩 What SmartWriter Does: Three Separate Powers, with the "Soul" Kept Under Extra Watch

The final design boils down to: build memory as three cleanly separated layers, each with a clear scope, clear ownership, and a clear home. The "style layer" in particular is where most of the customization lives. Here's the overall division of labor:

Now let's unpack the details inside each.

Personal writing style is the product's soul, and it's watched the most closely

The profile (the style layer) is the soul of the entire product, so it gets the strictest rules of all:

The writing agent itself is never allowed to modify the profile directly.

What does that mean in practice? No matter how confident the model "feels" mid-session that it's picked up on a new preference of yours, there is no tool it can call to write the profile file. Every change to the profile has to go through an out-of-band background process: wait until you've accumulated N pieces of writing, have a dedicated analysis job go through them offline and distill patterns, and even then, every single suggestion it produces has to be confirmed by a human — you click "accept" or "reject" on each one, individually, before it counts for anything.

Why so cautious? Go back to the bad case from the opening: style signals are inherently noisy. One article using an obscure word might just be what that piece needed — it doesn't mean you want your voice changed. Or you deleted a sentence the agent wrote, and that might just mean that one sentence didn't fit — not that you hate that sentence pattern in general. If the model were allowed to casually tweak the profile mid-write, as it goes, that noise would flow in continuously and contaminate the core — and the more it contaminates, the less it actually sounds like you. So I'd rather have profile updates be slower, dumber, and require your explicit nod than let them get casually edited inside the writing tool loop.

How does the profile actually get "into" the model's head?

The personal profile file sits on disk — the model can't see it there. Something has to route it in. During development, I went back and forth on a few approaches, and rejected two of them:

1. Inline the profile file directly with @import? No. @import is "unconditional, load-it-all" — it can't express "load only the layer of the profile relevant to the current genre, and resolve the priority order between layers." Writing an essay should pull the essay profile, not dump product-tech writing conventions in on top.
2. Inject it via a tool result? No, again. Remember the key insight from post 2: after compaction, what gets automatically re-run is hooks — not arbitrary tools. If the profile were injected via some tool's return value, it wouldn't get automatically restored after compaction, and the profile would just get summarized away. That would tie the product's soul to a thread that compaction can cut at any moment.

The approach that stuck: the moment a task starts, a module called profile_reader assembles the right profile for that piece's genre (essay? review? product-tech?), and writes it into a dedicated block inside CLAUDE.md. That gets the profile a free ride on the same mechanism CLAUDE.md already has — re-injected fresh on every new message, naturally immune to compaction.

As the diagram below shows, "profile_reader assembles it, writes it into a CLAUDE.md block" is the clean solution that satisfies both "resolve per genre" and "survive compaction" at once:

Why the fact layer gets split into two halves

Digging one level further, I did one more thing inside the fact layer: split it into two halves, each managed separately. Because there's a hidden tension inside "facts" too — some facts should be isolated, some should be shared.

• The kind that should be isolated (collection-level facts) → I manage a memory.md myself. Say you're writing a themed series — "what conclusion did piece 3 of this series land on, what stance did it take" is internal to that series, and shouldn't bleed into a different series you're also writing. I store it in a file inside the collection's own directory, relying on the filesystem's directory structure for natural isolation — collection A's memory physically lives in A's folder, and can't leak into B, with zero extra engineering required.
• The kind that should be shared (user-level facts) → hand it to the SDK's Auto Memory. Something like "you're a solo builder working on an AI product" is background every one of your writing tasks should already know. Auto Memory happens to be "shared per repo" — and here, that trait turns into a usable feature instead of a limitation.

To keep Auto Memory from overstepping (it's the model writing autonomously, and it might slip and record a style judgment too), I drew an explicit line for it inside CLAUDE.md: only record factual background, explicit corrections, and collaboration agreements — never record writing-style judgments; style belongs to the profile, full stop. Auto Memory does read that guidance in CLAUDE.md, but it can only ever act as a soft nudge, not an enforced boundary.

Writing to these two fact halves works differently from the style layer: style updates require human confirmation (it's the soul — handle with extreme care), while collection-level facts update automatically, out of band (facts are about timeliness first — you shouldn't have to approve every single one).

📐 Two Deep Dives: Keeping the Profile Uncontaminated, and Keeping the File from Bloating

Two threads left hanging above: exactly how does the style layer's out-of-band denoising work, and exactly what does the fact layer choose to record versus skip? Both, unpacked below as deeper detail.

📐 Going deeper · the style layer's three-part anti-contamination system: resolution, batching, reject memory

Since the profile is both the soul and inherently noisy, I built it three gates:

① Resolution priority (hard limits never get overridden): when multiple profile layers stack, they resolve in the order global hard limits > genre > global preferences. "Global hard limits" are non-negotiable (say, you've said "never use this kind of metaphor") — no more specific layer is ever allowed to override them. "Global preferences," on the other hand, are soft constraints, and a more specific genre layer is allowed to refine them. This guarantees that the lines you care about most always win, and never get quietly bent by some genre's temporary preference.

② Single-track batch processing + mandatory review: the profile doesn't get an update per article (way too jittery) — it accumulates N pieces (default 3, configurable), runs one analysis pass across all of them, produces a batch of "here's how we suggest adjusting your profile" items, and you accept or reject each one individually. There's only ever one pending batch at a time; a new trigger merges into the existing one instead of creating a second. Slower, but stable — you always keep your hands on the wheel.

③ Reject memory (recording what you don't want, too): suggestions you reject get written into their own small ledger (profile_feedback.json), keeping a rolling window of the most recent batch. Next time an analysis runs, those explicit "you said no to this" items get fed back in, so it stops proposing similar suggestions again. Treating negative feedback as an asset matters just as much: a rejection is a stronger signal than an acceptance, because "accepted" might just mean the user didn't bother to change it back, while "explicitly rejected" means they genuinely don't want it.

Every bit of care in the style layer serves one goal: make "gets better the more you write" actually true, instead of quietly drifting off course, pulled along by noise.

📐 Going deeper · restraint in the fact layer: not "remember everything," but "remember what breaks things if deleted"

I organize collection facts (memory.md) around 7 categories (series positioning, index of pieces already written, the series' stance and arguments, author background, likes/dislikes, explicit requirements, bird's-eye observations). But the real difficulty isn't the categorization — it's the inclusion criteria. Early on, I hit a real problem: I had it record "which arguments this series has already used," and one article alone could yield ten arguments — thirty articles, three hundred entries, and memory.md just exploded.

On review, the issue was a granularity mismatch: an argument is something internal to one article; what series-level memory should actually manage is "consistency across articles." Recording technical facts, mechanism explanations — the kind of thing a reader picks up just by reading the article itself — is pure restatement, pure bloat.

I eventually tightened the test down to one sentence: if you deleted this entry, would the next article end up repetitive, contradictory, or off-track because of it? If yes, keep it. If no, drop it.

By that test, only things that genuinely affect continuity going forward get recorded — "a stance the author has explicitly stated" (contradict it next time, and readers will notice), "an index of pieces already written" (prevents repetition) — and nothing else.

⚖️ Wrapping Up: The Engineering Hiding Behind "Really Getting You"

"Gets better the more you write" reads, to a user, as warmth. But making that actually true is entirely built on cold, unglamorous architecture decisions — separation of powers, denoising, resolution order. And it's exactly those invisible decisions that determine whether it genuinely understands you, or ends up like my first attempt: one big pot, where an offhand complaint gets solemnly filed away as part of your personality.

⚖️ The tradeoff · what separates generic and vertical, when it comes to memory

	The "lazy" default	What SmartWriter does	Why
Memory structure	One channel, record everything into it	Three separate powers: style / rules / facts, each its own drawer	Three fundamentally different things, mixed together, guarantees misfiling
Who can write "style"	The model, autonomously, whenever it feels like it	Profile only writable by an out-of-band worker + mandatory confirmation, SDK never touches it	Style is the soul — one round of noise contaminating it is fatal
Memory philosophy	Lean toward "remember as much as possible"	Only record "what breaks continuity if deleted"; never restate facts	Hoarding everything just means bloat and a collapsed signal-to-noise ratio — understanding you comes from judgment, not volume

Memory answers what the agent remembers. But we still haven't touched something even more fundamental: what "personality" does it actually operate with — what discipline does it hold itself to while it's working with you? Does it behave like a rigid, by-the-book engineer, or like a copilot with genuine tact?

That "behavioral foundation" isn't memory's job — it belongs to the System Prompt. Starting next post, we move into the "Steering the Model" section, and take apart the real personality switches on this machine. See you there.