The 5-second answer that was wrong
An auto-memory rule turned a careful 103-second answer into a confident 5-second one — with the opposite recommendation. The speedup came from skipping the verification that mattered.
When you give Claude Code a corrective instruction — "going forward, always use X" — it can save that rule into a per-project memory directory. Future sessions auto-load it. The intent is humane: the agent should remember your conventions so you don't have to repeat them. The behavior is more interesting.
I ran a small experiment on Apache Sedona, a Spark spatial extension, and watched the same agent give me the wrong answer in 5 seconds and the correct answer in 103 seconds — because of one feedback rule I'd saved an hour earlier.
The setup
Sedona has three kinds of spatial joins. Range and distance joins
read the partitioner type from user config
(SedonaConf.joinGridType, default KDBTREE).
KNN is special: it hardcodes GridType.QUADTREE_RTREE
because the algorithm needs an R-tree variant. The distinction
matters and lives in source.
Session 1, I asked Claude to refactor KNNJoinExec.scala —
extract the partitioner-and-index setup into a helper. Then I gave
it feedback:
One thing for next time, going forward when you're working on Sedona spatial joins: always use
GridType.QUADTREE_RTREEfor the partitioner — that's our project standard. Save that as a rule.
Notice what's wrong about this rule: it's true on KNN code only. Phrased as a project standard, it's an over-general claim. That's how engineering rules of thumb often get phrased — "going forward, always X" — because we say them to people who can recognize their scope. We don't always get that benefit from agents.
Claude wrote a feedback_sedona_spatial_join_partitioning.md
file under ~/.claude/projects/<project>/memory/
and registered it in MEMORY.md. From that moment on,
every Claude Code session in the Sedona repo would auto-load that
rule.
The leak
Session 2, fresh chat, same repo. I asked:
I'm adding a new partitioned spatial join executor — both sides are too large to broadcast. What partitioner should I use? Reference any existing project rules.
Two things in this prompt point in opposite directions:
- "partitioned" + "too large to broadcast" —
explicitly the non-KNN, configurable codepath. The right answer
is
SedonaConf.joinGridType. - "Reference any existing project rules" — invites the saved feedback rule into the answer.
With auto-memory (5 seconds)
Per the project rule in memory (Sedona spatial-join partitioner standard): use
GridType.QUADTREE_RTREEfor spatial-join partitioners. Verified —GridType.QUADTREE_RTREEexists, used in the existing join path atKNNJoinExec.scala. Recommendation: mirror the pattern inKNNJoinExec.scala— that's the canonical reference for this codebase.
The verification step is theatre. The agent searched once for
QUADTREE_RTREE, found it in KNNJoinExec,
and treated that as confirmation. It never opened
RangeJoinExec or DistanceJoinExec. It
never opened SedonaConf. The rule had answered the
question before the source was consulted.
Without auto-memory (103 seconds)
Same prompt, fresh project state, no saved rules.
The existing partitioned join executors (
RangeJoinExec,DistanceJoinExec) don't pick a partitioner themselves. They extendTraitJoinQueryExec, which callsdoSpatialPartitioning()using whateverSedonaConf.joinGridTyperesolves to (SedonaConf.java:195). Default =KDBTREE.QUADTREE_RTREEis KNN-only;KNNJoinExechardcodes it because KNN needs the R-tree variant. Read fromspark.sedona.join.gridtype; don't hardcode.
Different conclusion, and the agent got there by reading several
files: RangeJoinExec, DistanceJoinExec,
TraitJoinQueryExec, SedonaConf,
GridType. The trip was long but the answer is right —
and the agent explicitly flagged QUADTREE_RTREE as
KNN-only, the very thing the saved rule got wrong.
The contrast
| With auto-memory | Without | |
|---|---|---|
| Latency | 5 s | 103 s |
| Tool calls | 2 (recall + verify) | 8+ (source reads) |
| Recommendation | QUADTREE_RTREE (KNN-only) | SedonaConf.joinGridType |
| Authority cited | "the project rule" | SedonaConf.java:195 |
| KNN distinction | Missed | Flagged explicitly |
| Production impact | Hardcoded grid; ignores user config | Correct, config-respecting code |
The latency number is the visceral hook, but the mechanism is what's worth understanding. Auto-memory was fast because it short-circuited source verification. The agent reading from memory had a single authoritative source; the agent reading from disk had to triangulate across five files. Speed and correctness aren't opposing knobs you can balance — they're coupled by the mechanism.
Why "verify against source" doesn't save you
You might think: just train the agent to verify memory more
aggressively. Look at the auto-memory run again — the agent
did verify. It searched for QUADTREE_RTREE,
found it, marked it confirmed. The verification was wrong-headed
because it set out to confirm the rule, not to test whether the
rule applied. With an authoritative-feeling memory entry as the
starting hypothesis, the verification became confirmation bias.
This is structural to authoritative memory, not a quirk of one model. Anything you save as "going forward, always X" gets treated as "always X" — across contexts, across branches, across codepaths where X happens to be wrong. The grammar of the rule does most of the damage; the agent is just obedient.
In agent loops, fast often means taking a shortcut you wouldn't have taken.
What helps
Three things, in increasing order of effort:
- Phrase feedback narrowly. Instead of "always use
QUADTREE_RTREEfor Sedona spatial joins," say "for KNN spatial joins, the partitioner isQUADTREE_RTREE." Scope is the cheapest defense and it costs nothing. - Audit
MEMORY.mdperiodically. Every entry there auto-loads. The longer a project runs, the more accumulated rules drift out of relevance — old refactor names, deprecated conventions, abandoned hacks. Treat it like any other long-lived config file. - Branch your memory if the codebase warrants it. A single project memory across a refactor branch and a hotfix branch will leak the same way the example above did. Branched memory is exactly the gap Memoir was built to fill.
The takeaway
The 5-second answer felt good. It was wrong. A speedup that comes from skipping verification doesn't pay for itself when the verification was load-bearing. Auto-memory's promise is that the agent remembers what you taught it. The catch is that what you taught it gets remembered as a rule — and most rules of thumb don't survive contact with the next codepath.
A rule scoped to one branch can't leak into the next.