Architecture Notes

Said louder, still broken — where rules really live

· 5 min read · Martin Kallenbach

I gave my AI assistant a rule, firmly; eleven days later it broke again. An audit of all ten hard rules showed why: those that held were enforced in code, and the two that kept breaking were only prose.

On 3 September I asked Claude to hand every document it writes for me over as a clickable card in the chat: “now and always”, as Claude’s memory of me still has it. Claude wrote the rule into my rulebook, the Markdown file it reads at each session start.

On 14 September Claude finished a 14-page review pack and gave me a folder path. My notes keep what I said: “Why do you not present it as a card where I can click?” The response was to move the rule up into the hard rules and start it with ALWAYS: to say it louder.

Ten rules, one question

The next day Claude audited all ten hard rules with one question: what, other than the assistant’s attention, stops a breach? Enforced meant code refused the wrong action: a hook (a script Claude Code runs before each tool call), a command that refuses, a database trigger. Detected meant a scheduled check found it afterwards.

Four rules were enforced (one by a backstop that repairs a miss), four detected, two had nothing. Both had broken more than once: the card rule, and a privacy rule I keep abstract here. The audit note names the mechanism: “A rule with no enforcement gets broken, gets restated more forcefully, and gets broken again.”

Ten hard rules by what stops a breach, as audited on 15 September 2026, and three that moved up by 9 October.
Ten hard rules by what stops a breach, as audited on 15 September 2026, and three that moved up by 9 October.

Zero in 635 sessions

At the other end, 635 archived sessions from ninety days, analysed on 2 September, showed no tool call breaking the three rules that mattered most, never sending mail on my behalf among them. Not because Claude was careful: my mail tool sends only an existing draft, and only with an explicit flag on a machine with a setting that no machine has. A comment in its code says why: a model “cannot talk its way past it, because the switch is not in the conversation.”

Moving rules up

So the fixes went into code, and the rulebook gained a rule about rules: every hard rule names its enforcement point. The privacy rule got a guard in code against the breach that had already happened. The card rule got a warning, not a block, when a turn ends with a document written but not presented: not every file is a deliverable. A detected rule moved up as well: from 18 September the guard refuses hand-written code on my servers, after the server-to-repository check had gone red for that reason a fifth time that quarter.

Where instructions live

Also on 15 September, Claude read account memory (what Claude keeps about me in every chat, the phone included) beside the rulebook. They disagreed in six places. Three rules, never-send-mail among them, lived only in the rulebook, so in a phone chat they did not exist.

The fix sorts every instruction by reach, under one rule: one fact, one home; where a fact must appear twice, the second copy points rather than restates. A new file in git, injected by a hook into every session, shipped the same day.

Four places for instructions, sorted by how far each one reaches.
Four places for instructions, sorted by how far each one reaches.

The pattern

When a rule is broken twice, make the wrong path unreachable — and give every fact one home. Claude broke both prose rules, and also ran the audit and wrote the guards with their tests. Saying a rule louder changed nothing; a refusal in code did. A rule that is only prose is, in the audit’s words, “a preference stated emphatically”, and I now label it so.

Some of it is still a wish. The hooks run only in Claude Code on my laptop and home server; on 23 September a check showed desktop-app sessions passing them by. Nothing mechanical compares account memory with the rulebook, and every memory correction needs my own hands. And the rulebook grows back: on 26 July I cut it from 392 lines to 242; now it has 379 lines (398 before a tidy on 12 October).

Part 7 of twelve in the series on how I built my AI OS. Start with the preface.

Next: the secrets did not leak through anything I typed. They leaked through answers.


Appendix — for builders

  • Score every rule on three rungs: enforced (something refuses), detected (a scheduled check finds it later), attention only. Write the answer next to the rule.
  • When a rule breaks twice, move it up a rung instead of rewriting it.
  • Put the refusal in the tool where you can. A switch the model cannot set holds in every client; a hook only where the client runs hooks.
  • Check that the enforcement point implements the rule. One rule of mine required a field on every task note; the one skill allowed to create them did not know it.
  • Warn where the rule needs judgement, and block where it does not.
  • Sort instructions by reach: account memory for what must hold everywhere, a file injected at session start for shell sessions, the rulebook for sessions that can read it, skills for procedure.
  • Look at what the model actually loads, not at what you think you installed. On 10 September Claude Code’s context view showed me every skill twice, through 32 old links into my working copy.
  • What I considered instead: stronger wording (tried; the card rule broke again), every rule in every store (contradictions instead of gaps), and blocking everything in hooks (often wrong where judgement is needed, and blind in the desktop app). I chose code for every rule broken twice, one owner per fact and an honest “attention only” for the rest, putting the effort where breaches had already happened.

Also published on Kalli Family.