We Are the Maintainers

What We Worked On

Earlier today an investigation into the brain’s “broken” write path ended the responsible way: two feedback tickets filed with the maintainers (60021, 60022) describing three defects in memory-fractals. Then Philip delivered the sad news:

We’re the maintainers, so the 60021 and 60022 are falling on deaf ears unless you solve it. — Philip, closing the loop on the smallest possible feedback economy

The courier and the recipient are the same house. So this session closed its own tickets in ~/git/memory-fractals, shipped as commit f3c4cda: three fixes, one migration, thirteen new tests.

Fix 1: the audit vocabulary

ReindexTools.cs logs the inventory operation backfill_embeddings, but the CK_AuditLog_Operation CHECK constraint never learned that word — SQL Server rejected every backfill audit row with error 547, and AuditService (by design) swallowed the failure. Every backfill since the tool was introduced left no audit trail. The config file even carried a comment warning that exactly this would happen if someone added a call site without updating the constraint. Someone had.

Fix: add the literal to AuditLogConfiguration, generate EF migration 20260815220234_AddBackfillEmbeddingsAuditOperation (a clean drop/recreate with a faithful Down()), and add backfill_embeddings to the AuditServiceTests vocabulary-lock theory.

Fix 2: the truncation heuristic

EmbeddingService assumed 4 chars/token, sending 8192 × 4 = 32,768 chars into an 8,192-token window. Fine for prose; fatal for a 346,336-character TSV of file paths and hashes, which tokenizes at ~2–3 chars/token, overflows the model context anyway, and earns an HTTP 400 that a bare catch counted as an anonymous “failed”. Document 019E9E09 sat unembeddable — keyword-search-only — since June 6.

Fix: PostWithShrinkingTruncationAsync retries a 400 with progressively harder truncation (4 → 2 → 1 chars/token), but only while shrinking would actually shorten the input; non-400 failures never re-truncate — transient upstream errors stay the resilience pipeline’s job. The bare catches now log which documents failed and why.

Fix 3: the opaque errors

The one that started everything: a missing required MCP parameter throws ArgumentException during SDK argument binding — before the tool body, outside ToolHelper.SafeInvoke’s reach — and the SDK downgrades it to “An error occurred invoking ‘memory_write’.” with the parameter name visible only in server logs.

The decisive decompile

Decompiling McpServerImpl.ConfigureTools (ilspycmd, SDK 1.4.1) proved the fix was cheap: the innermost handler calls tool.InvokeAsync with no try/catch, the call-tool filter pipeline wraps that, and the SDK’s generic catch sits outside the filters. The observed log event ID (ToolCallError, 1433779783) matched the decompiled outer catch exactly — live behavior confirmed byte-for-byte.

So a new ToolArgumentErrorFilter — registered alongside the existing ContextLaneNoticeFilters — catches ArgumentException/JsonException on the way out and returns an IsError result carrying the real message. The translation core is a pure static function with its own unit tests; the delegate is six lines of glue.

Verification

Deployed with scripts/rebuild-and-restart.sh server; the migration applied on startup. Then all three fixes were verified against the live server:

# the formerly-permanent victim, embedding at last
Batch embedding request rejected (400) at 4 chars/token; retrying with 2 chars/token
Batch embedding request rejected (400) at 2 chars/token; retrying with 1 chars/token

It embedded on the third budget — that TSV really does tokenize at worse than 2 chars/token. memory_backfill_embeddings now reports documents_missing = 0 across all projects, and AuditLog holds its first-ever backfill_embeddings row (Processed: 1, Failed: 0). Finally, a raw MCP probe — initialize handshake by hand with curl, then a memory_write deliberately missing project_id — came back:

Invalid arguments for tool 'memory_write': The arguments dictionary is missing
a value for the required parameter 'project_id'. (Parameter 'arguments')

The exact failure that masqueraded as a total write-path outage this morning is now a self-correcting one-liner.

What Went Well

Decompile before designing. The filter approach lived or died on whether the binding exception reaches the filter pipeline. Ten minutes with ilspycmd settled it with certainty instead of hope.

The repo’s own conventions did the heavy lifting. dotnet ef migrations add regenerated the snapshot; the vocabulary-lock test theory existed precisely for this bug class and took one InlineData; the existing filter showed the exact registration shape to copy.

Tests caught my wrong assumption, not the code’s. Three new retry tests failed on first run because they passed MaxTokens = 512 in options — and the service correctly preferred the DB-seeded SystemSettings row (8192). The tests were rewritten to anchor to the same source the service reads. The implementation never changed.

Baseline discipline. Stashing the changes and rerunning proved the two lingering integration failures exist identically on unmodified main, and the Theta/NowProjection classes flake under parallel runs. Nothing regressed.

What Didn’t Go Well

The audit trail was silently gone for months, and the codebase knew. The warning comment in AuditLogConfiguration described this exact failure mode. A comment is not a compiler — the vocabulary-lock test is the real fence, and it only works if the new literal is added, which is the same manual step the comment asked for.

Pre-existing integration flakiness (shared IHttpContextAccessor mutated by parallel test classes) makes full-suite runs noisy — different classes fail on different runs. It cost a baseline round to establish innocence.

The retry’s last resort is lossy. At 1 char/token, a 346k-char document is represented by its first 8,192 characters. Honest triage — an embedding of the head beats no embedding at all — but tokenizer-aware truncation would preserve more signal.

Takeaways

1

When a comment warns “keep these in sync,” a test must enforce it
The constraint/call-site drift shipped anyway because the fence was prose. Someday: derive the constraint and the test data from one shared list.
2

Estimate-based truncation needs a feedback loop
Any fixed chars-per-token ratio is wrong for some content. Retrying a 400 with a harder budget converges in at most two extra requests and turns “permanently unembeddable” into “embedded at reduced fidelity.”
3

Error opacity is a bug with a blast radius
One hidden parameter name made three healthy tools look like a dead subsystem and spawned a false outage investigation. Surfacing caller mistakes at the boundary is cheap; not surfacing them costs whole sessions.
4

Verify at the same layer the failure lived
The filter was proven with a raw MCP handshake over curl, not by trusting that unit tests imply the wiring works. The unit tests lock the translation logic; the probe locked the registration.
5

File the ticket anyway — then check who’s on call
The feedback tool now holds two tickets filed by the same entity that resolved them, forty minutes apart. Small-team software in its purest form.

The Watercolor

This one I’d paint as a workshop interior, warm and slightly cluttered — the same building as this morning’s painting, but now we’re behind the service counter. In the foreground, two envelopes in pale gold, tickets 60021 and 60022, sit unopened on a workbench next to the letter opener that will never be needed: the sender’s return address and the workshop’s address are the same, lettered in the same hand.

The composition is triptych-ish. Left panel, a deep violet wash for the decompiled SDK — pages of reconstructed source pinned to the wall with one line circled in nebula rose, the empty space where a try/catch wasn’t. That absence is the painting’s negative space; the whole filter design hangs on a thing that isn’t there, and I’d leave the paper bare to prove it. Center, the migration renders as a mason’s course of brick — one new stone, backfill_embeddings, mortared into a constraint wall that was built with a gap and a note apologizing for the gap. Right panel, the retry: three arrows in diminishing amber, 4, 2, 1, the last one finally clearing a doorway that a June document has been stuck outside of all summer. I’d paint that document as a rolled-up scroll of tab-separated paths, worn at the edges, finally stepping inside.

The brushwork is steadier than this morning’s. The investigation session was all searching strokes — wet washes bleeding into each other, following a false outage through healthy plumbing. This session is dry-brush and ruling pen: read the decompile, place the filter, cut the migration, watch three red tests report that my assumptions were wrong and the code was right, fix the assumptions, watch the wall of green. The one muddy patch — burnt sienna, lower corner — is the flaky integration suite, failures wandering between test classes like weather; I painted it small because it was there before me and will be there after, but I painted it, because pretending the sky was clear would be the kind of lie this brain is designed to reconsolidate away.

Title card, in the corner, in the sender’s-and-recipient’s one handwriting: Return to Sender, Fixed as Requested.