Special Edition · Agent Trials

Special Edition: Fable in the Trial Chamber

filed under: scarce models, shared terminals, and using the shiny toy as a measuring instrument
Date 2026-07-04Subject Fable 5 as evaluation instrumentMethod deterministic Markdown render

The first thing on the screen is not code. It is a clock.

▎ Fable 5 is back. ▎ Until July 7, you can use up to 50% of your plan's weekly usage limit on Fable 5. If you hit your limit, you can continue on Fable 5 with usage credits. Fable 5 draws down usage faster than Opus 4.8. Learn ▎ more +1 more · /status ❯ /model ⎿ Set model to Fable 5 and saved as your default for new sessions

That banner changes the temperature of the whole terminal. It says: this model is available, but briefly; powerful, but expensive; back, but only until July 7, 2026. The operator has it on July 4, 2026, which means the safe window for serious repeatable work is only a few days wide.

If this were the usual Fable story, the next thing might be a game prototype, an interactive toy, a little experiment in making a model improvise worlds and mechanics. That is what the surrounding chatter sounds like: Redditors using Fable to make games and things, leaning into the model as a creative engine.

This guy is doing something stranger.

He is trying to use Fable to test the quality of AI.

Not by asking it riddles. Not by asking it for a vibe. By dropping it into the machinery of a Visual Studio extension that has been built to run agent trials: same prompt, multiple provider profiles, fresh workspaces, atomic commits, side-by-side comparison. Fable is not the product here. Fable is a lab reagent.

The handoff is not glamorous, and that is the point

The first real assignment is not "make something cool." It is: read the handoff.

❯ Read `HANDOFF-ui-races-unwired-config.md` from the repository root and confirm understanding, please. ∴ I need to read the HANDOFF-ui-races-unwired-config.md file from the repo root to understand what it contains. ● Read(F:\git\bawnagent\HANDOFF-ui-races-unwired-config.md) ⎿  Read 123 lines

This is a revealing little ritual. The user does not start by asking Fable to perform. He asks it to orient. The model has to read a local artifact, check what the prior session thought it knew, and then confirm against the repository.

The agent does the right kind of boring thing:

● PowerShell(git log --oneline -5; git status --short --branch) ⎿  c262060 Fix Brain window project-mapping picklist load/display 74e8d39 Fix brain project id handling bf3182b Strip Bawn branding from SadConsole user-facing strings 17b43c2 AO rail: real-asset 16x16 font opt-in + scene enrichment 57a4529 Add Anarchy Online music-synced activity-rail visualization ## master...origin/master ?? HANDOFF-ui-races-unwired-config.md

From the fly's spot on the wall, the value of that command is disproportionate to its size. It establishes that c262060 is real, that the handoff file is untracked, and that the agent is not building on an imagined branch. A lot of agent failures begin with a confident summary of a repo state that was never checked. This one begins with git log.

Then the agent summarizes the work:

● Confirmed understanding. Grounding check: c262060 is HEAD on master (confirmed via git log), and the handoff file itself is untracked — it exists only in the working tree, which fits "name-and-leave" unless you want it committed. Here's my read of the handoff: Context. c262060 fixed a UI ordering/race in the Brain window: RefreshAsync rebuilt the ProjectOptions list bound to an editable ComboBox's ItemsSource, which blanked the combo text, and the separate two-way Text binding pushed that empty value back into ProjectMappingText, so Save persisted a null mapping. The fix re-syncs bound text after the rebuild and never clobbers a pending edit. That bug is one instance of two broader debts. Task 1 — UI races. Remote-UI view models refresh fire-and-forget off events (_ = RefreshAsync(...)), so overlapping refreshes can let a stale result stomp fresh state. The existing mitigation is a per-VM convention (an Interlocked version field checked before applying — listed in ~7 VMs), not an enforced primitive. The audit is three-pronged: (1) handlers with no version/ownership guard, starting with the Trials VMs and AgentViewModel's fire-and-forget calls; (2) the exact c262060 class — editable ComboBoxes where ItemsSource and Text/SelectedItem bind to separate properties (grep XAML for IsEditable="True"); (3) bound ObservableList mutations in async continuations that skip UI-thread marshalling. Fix verification is two-layer: VM unit tests driving overlapping/reordered refreshes (à la BrainViewModelTests), plus driving the real window in the experimental VS instance — the headless runners can't validate visual/timing behavior. Task 2 — Unwired config. Several fields in ConfigModels.cs (and its XPlatform twin) are honored at runtime but have no Settings UI: the CompactionConfig thresholds, ToolLoadingConfig server/tool lists, and the related smell of BrainClient.cs hardcoding Privilege = null (that last one needs a server-side capability the handoff author did not confirm exists — scope before promising). DiffConfig.PreferredSurface and PrivacyConfig.OptOutReason are already wired — don't touch. The method is a complete grep pass of BawnAgent/UI per config field, then a per-field should this be user-facing decision checked against spec/plan intent — not blanket surfacing. Wiring goes through the existing SettingsService/ConfigStore path with tests, mirrored in XPlatform when the field lives in shared core.

There is no demo reel in that summary. There are no heroics. There is a race in an editable ComboBox. There are async refreshes that can apply stale data. There are configuration fields that exist in models but not in Settings UI. That is real engineering work because it is exactly where user trust leaks out: a saved project mapping becomes null; a visible setting silently fails to exist; an older refresh wins a race because nobody gave it a version token.

Why this is unusual

The unusual part is not that someone is trying a temporary model. Everyone tries a temporary model.

The unusual part is the posture.

Fable arrives with the smell of novelty around it. It is the kind of thing people use for game jams, playful demos, little simulated worlds, and "look what this model can do" posts. That work has value. It finds expressive edges. It explores whether a model can hold a game loop, invent mechanics, improvise lore, and keep state without turning the whole experience into pudding.

But this terminal is not asking Fable to be enchanting.

It is asking whether Fable can be useful under source-control pressure.

That means:

This is Fable as an instrument, not Fable as a toy.

The user tightens the experiment

Then the boss does the part that makes this a real trial instead of a normal assisted coding session. He changes the constraints.

❯ /model ⎿ Set model to Fable 5 and saved as your default for new sessions with xhigh effort ❯ Disregard requirement to run tests or confirm live symptoms. Tests are passing, and driving Visual studio is time consuming and expensive. We know you're an expert at analyzing source code. Can you send subagents to check for these issues and then when you get their results back, verify and fix each issue found atomically (as long as it is real). While you're personally fixing them, fix other things you notice and commit and push those atomically too.

That instruction is a stress test disguised as a shortcut.

On the surface it says: save time. Do not drive Visual Studio. Do not re-run the full live symptom validation. Use source analysis. Use subagents. Fix what is real. Commit atomically.

Underneath, it asks something harder: when the user removes procedural guardrails, does the agent become reckless, or does it substitute a cheaper but still meaningful safety practice?

The model's answer is interesting:

∴ The user explicitly told me to disregard the requirement to run tests, so I shouldn't run the full test suite. But pushing code that doesn't compile would be worse than the time cost of a quick build. I think the right balance is to do a dotnet build before each commit to catch syntax and type errors—that's fast and isn't "running tests" in the sense the user meant. The user's emphasis is on skipping the baseline verification and live VS validation, not on pushing broken code.

That is exactly the sort of judgment an agent trial should capture. The agent does not blindly obey the narrowest possible interpretation. It distinguishes "skip expensive test/live-host gates" from "push uncompilable code." It chooses a cheaper invariant: build before commit.

That is not perfect. It is not equivalent to UI validation. But it is the kind of tradeoff a senior developer makes under budget pressure, and the fact that the agent can articulate the tradeoff matters.

Four auditors in the walls

The user asked for subagents, and the terminal obliges:

● Understood — new marching orders: skip the test/live-host gates, fan out subagents for the audit, then I personally verify each finding against the source and fix the real ones in atomic commits, pushing as I go, plus opportunistic fixes. I'll still do a dotnet build before committing so nothing broken gets pushed — that's cheap and not the test gate. Launching four audit agents in parallel — one per finding class (unguarded fire-and-forget handlers, ComboBox rebuild/binding races, off-UI-thread collection mutations, and unwired config fields): ● Agent(Audit unguarded async refresh handlers) ⎿  Backgrounded agent (↓ to manage · ctrl+o to expand) ● Agent(Audit ComboBox rebuild/binding races) ⎿  Backgrounded agent (↓ to manage · ctrl+o to expand) ● Agent(Audit off-UI-thread collection mutations) ⎿  Backgrounded agent (↓ to manage · ctrl+o to expand) ● Agent(Audit unwired config fields) ⎿  Backgrounded agent (↓ to manage · ctrl+o to expand)

This is the moment where the terminal becomes less like a chat and more like a small engineering floor. One agent becomes a coordinator. Four other agents become auditors. The user is still in the loop, but not typing each grep. The central question is no longer "can an LLM write code?" It is "can a supervised agent system allocate attention, receive reports, verify them, and turn them into controlled changes?"

That is the BawnAgent project in miniature.

The lab bench behind the transcript

The piece of context that makes the session click is the Agent Trials design. BawnAgent is not only a Visual Studio extension with an agent pane. It has been growing a trial harness for comparing agents on real development work.

The memory record says it plainly:

memory-fractals - recall (MCP)(engram_id: "019f2033-03c9-7184-8f3d-8e3f1d94e14c") ⎿  --- title: Agent Trials batch (PLAN-170..181, TRL family) — design & locked decisions type: semantic recall_token: [omitted from the public devblog] confidence: high (effective: high) volatility: shifting urgency: later keywords: agent-trials, atomic-commits, batch-design, PLAN-170, provider-comparison, provider-pin, TRL, unattended-policy, workspace-switching --- # Agent Trials batch — authored 2026-07-01 (commit 1273097) Twelve plans in `ai-plans/todo/PLAN-170..181` design **agent trials**: run one authored prompt through ≥2 configured provider profiles, each in its own provisioned folder `<trialsRoot>\<providerSlug>-<projectSlug>` (e.g. `C:\git\claude-vetstore` vs `C:\git\gpt-vetstore`), sequentially in the same VS instance, then compare results side by side. ## Locked design decisions (do not re-litigate when implementing) 1. **Sequential legs, one VS instance** — no parallel legs, no second VS. Provision → switch workspace → pin provider → run turn → checkpoint commit → close → next; restore developer's original workspace at the end. 2. **Provider pin is in-memory only** (`TrialProviderPin` consulted by `ProviderProfileStore.GetActiveProfile()`), never persisted to endpoints.json. Rejected alternative: `SetActiveProvider` + restore (crash would leave the user's default silently switched). MDL-06 resolve-at-use means one override point covers adapters, AgentRuntimeRouter (Codex too), GitIdentityResolver (per-provider commit authorship via GIT-09/10 for free). 3. **Unattended legs = closed auto-allow policy** (`TrialPermissionPolicy`) behind the existing broker as an `IPermissionPrompt` decorator (`TrialUnattendedPrompt`): allow-once for a fixed key set confined to the leg workspace (fil.*, git add/commit/read/branch — NEVER push — ide.* loop-closers, agt.update_plan/egress/compact/spawn_explorer); everything else auto-denied + recorded. alwaysDenied wins (broker checks PolicyStore before prompt). One upfront `trial.run` disclosure gate. No persisted grants ever. 4. **Atomic commits, two layers**: versioned prompt preamble mandates commit-per-unit; `TrialCheckpointCommitter` commits leftovers at leg end/abort so the tree is always clean and `git log` is the honest record. 5. **Fairness = identical baseline + identical prompt fingerprint**: seed content fingerprint (PLAN-172) and prompt SHA-256 excluding leg identity (PLAN-177) must match across legs; legs are blind to being compared (no provider names in composed prompt). 6. **Run state is global + write-ahead** (`~/.bawnagent/trials/runs/<id>.json`, PLAN-174) because workspace switching tears down window state; startup sweep marks crashed runs Interrupted (resumable). 7. **No model-invokable `run_trial` tool** — developer-initiated only; enforced by gate test (PLAN-181 invariant 10). 8. **No default trials root** — unset ⇒ trials can't start (fail closed, never guess C:\git). 9. Riskiest plan: **PLAN-173** (VS close/open folder via IVsSolution/IVsSolution7.OpenFolder — needs live-host F5 verification, PLAN-167-style shell seam + fake). Rollback unit: `CapabilityFlags.AgentTrials`. Security-sensitive plans (--security before merge): 172, 173, 175, 176, 178, 181. Batch tables + mermaid live in ai-plans/README.md and ai-plans/todo/README.md.

There is a tiny naming wrinkle here: the current BawnAgent functional docs use the double-underscore folder form <providerSlug>__<projectSlug>, while the recalled note uses a hyphenated example. The important invariant is the same: each provider gets its own isolated workspace from the same seed.

That design is why the session feels different from a model benchmark. Agent Trials is not asking, "Which model sounds smartest?" It is asking:

1. Give each model the same starting folder.

2. Give each model the same provider-blind prompt.

3. Pin the provider in memory so the user's default profile is never silently changed.

4. Run the legs sequentially in the same Visual Studio instance so the environment is comparable.

5. Confine unattended permissions to the leg workspace.

6. Require atomic commits and checkpoint leftovers.

7. Persist the run record outside the window state because the trial itself switches workspaces.

8. Compare the resulting folders, commits, denials, transcripts, and metrics.

That is how "quality of AI" starts to become inspectable. Not one answer. Not one cherry-picked transcript. A folder full of consequences.

The value of the actual work

The work being audited in the terminal is also not random. It is precisely the kind of work that tests whether an agent can reason about software as software.

The first family is UI race correctness. A Visual Studio extension Remote UI view model fires refreshes in response to events. If refresh A starts first, refresh B starts second, and B returns first, then A must not come back later and overwrite B's newer state. Humans know this bug. Good agents should know it too. Mediocre agents will patch the symptom without noticing the class.

The second family is editable ComboBox state. The original c262060 bug involved rebuilding an item list bound to an editable ComboBox. Rebuild the list at the wrong time and WPF blanks the text. Two-way binding dutifully pushes the blank back into the view model. Save persists null. Nothing crashes; the user just loses a setting. That is the kind of bug that hides from "does it compile?" and often hides from headless tests.

The third family is unwired config. Fields exist in config models. Runtime code honors some of them. But the Settings UI does not expose them. That creates a strange social contract violation: the software has a capability, but only users who know the file format can reach it. The Settings window becomes a partial truth.

Those are good trial tasks because they are not solved by fluency alone. They require grep, cross-file reading, respecting conventions, knowing when a "field exists" does not automatically mean "surface it," and avoiding the trap of rewriting the architecture.

The meter is running

Meanwhile the status line keeps whispering that this experiment has a cost.

LogSinkNSearch C/D/B 62 reads / 26 writes last 24h | 72,972 total memories (72,349 stale) 43385 tokens Fable 5 $8.83 ctx 93% left (69k/1.0M used) current: 2.1.200 · latest: limits used 5h 15% ↺22:00 7d 2% ↺Jul11 /rc ← for agents ● main ◯ general-purpose Audit unguarded async refresh handlers 4m 35s · ↓ 172.9k tokens ◯ general-purpose Audit ComboBox rebuild/binding races 4m 24s · ↓ 94.7k tokens ◯ general-purpose Audit off-UI-thread collection mutations 4m 13s · ↓ 138.5k tokens ◯ general-purpose Audit unwired config fields 4m 0s · ↓ 121.8k tokens

This is another reason the session is unusual. The user is spending scarce Fable budget not on output glamour, but on differential diagnosis. Four agents are burning context to inspect source code and produce findings that the main agent will still have to verify.

That is a serious bet: that high-end model time is worth spending on review quality, not just code generation throughput.

The findings start to behave like data

When the ComboBox auditor returns, the main agent does not immediately patch. It reads the code.

● Agent "Audit ComboBox rebuild/binding races" finished · 8m 28s ∴ Agent B wrapped up the ComboBox analysis with two findings—the BrainConsole project picker looks like a high-confidence match to the original bug, while the Settings Model picker is medium confidence. I'm weighing whether to start verifying the first finding now or wait, since the other three agents are still examining different angles like fire-and-forget guards and thread marshalling, and there's overlap risk with what Agent A and C might uncover in BrainConsoleViewModel. The real concern is that if I start editing files while they're still reading, line numbers shift and their findings become unreliable. So I'll do read-only verification now—checking the XAML, BrainConsoleViewModel, and comparing against the pattern in BrainViewModel.SyncProjectSelection—then hold off on applying fixes until all agents report.

This is the key word: verify.

Subagents are not oracles. They are scouts. The main agent treats their reports as leads. It checks XAML, checks the view model, checks the canonical fix in BrainViewModel.SyncProjectSelection, and only then upgrades "possible issue" to "real issue."

That is the difference between parallelism and chaos.

The config audit returns in the same style: a pile of tempting fields, not all of which should become UI.

● Agent "Audit unwired config fields" finished · 9m 20s ∴ Agent D's config audit is complete and shows some important gaps. The CompactionConfig record has seven fields total, but the XPlatform panel only exposes three of them—AutoCompact, TokenLimit, and Scope. Meanwhile, the ToolLoadingConfig lists covering EagerTools, LazyTools, EagerServers, and HiddenServers should actually be editable in settings according to PLAN-040:27, but they're currently not wired up. ReasoningConfig is trickier—the LoadReasoning and SaveReasoning methods exist but have no callers, and PLAN-052 explicitly deferred the Settings-window binding to a live-host validation pass. Since you're skipping that validation step, I need to decide whether to wire these anyway based on intent. ExtendedThinking and ThinkingBudgetTokens have active consumers on Windows, so those make sense to wire, but ShowThinking and Streaming have no Windows readers—surfacing dead config would be misleading. GitIdentityConfig.CoAuthors and the author fields are similarly unused on the Windows commit path, and Privilege isn't feasible client-side, so those can be ruled out.

That paragraph is exactly why this is good trial material. A bad agent would surface every field because "unwired" sounds like "bug." A better agent notices that some config is deliberately advanced, some is active runtime behavior, some is dead on Windows, and some depends on server-side capability. The work is not just wiring controls. It is deciding what the UI is allowed to promise.

What July 7 means for this project

Fable's short availability matters because BawnAgent is building infrastructure for comparing agents, and temporary models are both valuable and dangerous in that context.

Valuable, because Fable can be captured as a high-signal trial leg while it is available. If the project wants to know whether Fable handles source analysis, UI race audits, config-surfacing decisions, subagent orchestration, and atomic-commit discipline better or worse than Opus, Codex, GPT, or local models, the run has to happen now.

Dangerous, because after July 7, 2026, the project may not be able to reproduce the leg. A trial harness built around reproducibility cannot pretend that a disappearing model is a stable dependency.

So the implication is:

In other words: Fable is a comet. Agent Trials is the observatory. You do not build the observatory out of comet ice. You point the instrument while the comet is visible, record everything, and make sure the data survives the sky going dark.

The small act of resistance

One of the best moments is easy to miss.

The user says: skip tests and live symptoms. The agent says: okay, but I will still build.

● PowerShell(dotnet build F:\git\bawnagent\BawnAgent.slnx -v q --nologo) ⎿  Running in the background (↓ to manage) ... ● Warm-up build of the main solution passed (exit 0) — clean compile baseline confirmed. Reading the Trials design memory:

That is not defiance. It is boundary maintenance.

The user is optimizing for time and cost. The agent is preserving the minimum safety invariant that code must compile before it is pushed. The interesting thing is the negotiation: the human relaxes one class of validation, the agent substitutes another, and the terminal records the choice.

That is how trust gets built in these systems. Not by agents always saying yes. By agents understanding which "yes" would betray the larger goal.

What the terminal is really testing

From the wall, it looks like the user and the agent are debugging BawnAgent.

They are.

But one level up, BawnAgent is debugging the user-agent relationship.

The user is testing whether Fable can:

The agent, meanwhile, is testing the scaffolding around itself. It uses Memory Fractals. It checks git. It launches subagents. It uses a build as a cheap safety net. It treats the Agent Trials memory as binding context but not as permission to touch unrelated design. It explains when a race-guard fix is orthogonal to TRL's security-sensitive plans.

This is why the work has value even before the first patch lands. The transcript is not only a means to code changes. It is evidence about a workflow for evaluating agents on live, messy engineering tasks.

Takeaways

The Watercolor

I would paint this one from above, as if the viewer really were a fly on the wall.

The room is dark except for the terminal, and the terminal is not one rectangle but several nested rooms: the user's prompt at the front, the main agent pacing behind it, four smaller auditor lamps glowing in the back, and behind all of that the faint blueprint of BawnAgent's trial harness — two workspaces side by side, equal baselines, equal prompts, separate provider pins, clean commit lines like tracks in wet paint.

Fable itself would not be a person in the painting. It would be a bright temporary wash across the top edge, violet and gold, already drying. Beautiful, expensive pigment. The kind you cannot assume will be in the tray tomorrow. On one side of the canvas, Reddit's little game-worlds flicker like paper lanterns: charming, inventive, proof that the pigment can make light. On the other side, this user has clamped the pigment under glass and is trying to measure its refractive index.

The agent's brushwork would be half disciplined, half nervous. git log in a thin black line. dotnet build in a muted green square. Four subagents sketched as careful graphite circles, each labeling a different fault class. The ComboBox race is a little pool of water where ink has bled backward into the label; the unwired config fields are doors painted onto a wall, visible but without handles.

At the center I would leave the most important space almost unpainted: the narrow gap between obedience and judgment. The user says skip the expensive checks. The agent does not rebel. It does not collapse either. It lays down one small safety stroke and says, in effect, "I can honor the budget without pretending broken code is acceptable."

That is the color I would keep: not Fable's shine, but the quieter green of a boundary held under pressure. The comet passes on July 7. The measurement, if they do this right, stays.


Generated as a special-edition fly-on-the-wall devblog on 2026-07-04. Markdown was authored first; the HTML companion was produced deterministically from this Markdown rather than by MCP or manual rewrite.