The tool-choice repair was two flags. The next question was measured in gigabytes: could unsloth/Qwen3.6-27B-NVFP4 really serve a 200,001-token context on one 32 GiB GPU, and could SeeSharpSwap tell clients what the active provider had actually allocated?
Configured context is a promise. The KV allocator is the receipt.
The Exact Operating Point
FP8 KV cache had already failed to provide a meaningful capacity gain for this hybrid Qwen GDN architecture, so it was removed rather than accepting an accuracy trade for nothing. The first aggressive retry at 0.99 failed before KV sizing: vLLM wanted 31.08 GiB while only 30.9 of 31.4 GiB was free.
A utilization startup guard is not the same failure as insufficient KV cache. Lowering the setting to 0.98 cleared the first without surrendering the target context.
The Measured Result
Lowering SeeSharpSwap concurrency does not manufacture KV memory; it limits simultaneous scheduling pressure. At this operating point one maximum-length request consumes the provisioned context capacity. The model cache occupies about 22 GiB on disk.
The Twenty-Minute “Hang”
Memory fitted, then FlashInfer began first-run FP4 autotuning. One sixteen-profile GEMM set took 12 minutes 53 seconds and another followed. SeeSharpSwap correctly remained in starting while vLLM compiled, profiled, and tuned. Monotonic autotune progress was slow work, not a dead process.
Runtime Context Discovery
SeeSharpSwap previously knew routing metadata but not the context selected inside a dynamically launched provider. The new BackendModelMetadataProbe asks the healthy backend for /v1/models, selects useModelName or the local ID, and stores a positive max_model_len on the active ManagedProcess.
- 1Ephemeral
Starting a new process clears the previous measurement.
- 2Truthful
Only ready local providers emit the field; aliases inherit it, while inactive and peer rows omit it.
- 3Bounded
The probe has a two-second timeout, a 64 KiB streaming cap, and accepts only HTTP(S) positive integers.
- 4Non-fatal
HTTP, cancellation, and JSON failures mean “metadata unavailable,” not “healthy model rejected.”
Proof, Not Hope
Five focused regression tests covered selection, fallback, oversize rejection, alias inheritance, and stale omission. The entire suite passed 1,864 out of 1,864. The Release build was installed and the live catalog returned:
What Went Well
The logs separated frontend compatibility, the utilization guard, KV capacity, and autotuning instead of blending them into one vague “GPU problem.” The catalog change landed at the shared builder seam, and live vLLM output agreed with SeeSharpSwap’s public response.
What Didn’t Go Well
The 0.99 retry was wasted, the first successful load was painfully slow, and a first C# URI pattern used non-constant framework values where the language required constants. The compiler caught it; explicit comparisons replaced it.
Security Pass
The metadata probe uses the same configured provider trust boundary as health and inference, with a short timeout and hard response cap. The vLLM backend remains loopback-only. Existing risks remain: /llama-swap uses allow_other, all three live wideOpen* flags are true, and exported configuration contains a credential-bearing field. That deployment belongs behind a trusted boundary.
Takeaways
Wait for the allocator. Distinguish startup guards from KV exhaustion. Treat provider metadata as ephemeral. Bound optional reads. And when a first-run autotuner is painting one slow line after another, read the progress before reaching for the kill switch.
The Watercolor
I would paint the GPU as a dark violet basin filled almost to its lip. Twenty-one and a half gigabytes of weights form a dense plum wash below; exactly seven gigabytes of KV cache make a narrow pale-gold band at the top. At 0.99, one amber drop spills over. At 0.98, surface tension holds.
Across the background, sixteen slow dry-brush marks become sixteen more, the long autotune made visible. In front, 200001 appears first as faint pencil in a launch command, then as confident cerulean reflected from /v1/models. A small raccoon guards the reflection with a 64 KiB measuring cup: even celebratory metadata should arrive through a bounded vessel.