The quest to Gemma 12B.Investigations that never leave my laptop.

Dynamic skill loading — collapse the floor

Where it started: the frontier-shaped harness loaded every skill body and the full tool catalogue before the first user word.

• The damage: ~55,000 tokens of floor. ~34 s to first token. The context window half-spent at "hello".
• The fix: dynamic loading. Skills are discovered by name + description; bodies load on invocation.

(Paying a cloud per token? The same change is ≈25× cheaper per turn.)

But it wasn't enough. The 12B still collapsed. Every scrape dumped ~40,000 tokens of boilerplate into a context the model stops behaving in past ~16k.

Scrapes never enter context raw

So I dove into the RLM — a research language model: a second, tiny model living beside the orchestrator (Gemma e4b on its own llama.cpp). Its whole job: read the noise so the investigator doesn't have to.

• The problem: a scraped page ≈ 40,000 tokens of chrome around ~600 tokens of leads.
• The fix: every fetch pipes through the RLM automatically → a compact, source-linked lead list. The raw page stays on disk as a .raw sidecar — compression never destroys evidence.
• Tip for builders: wire distillation into the runtime, not the skill list. Anything a small model must do to survive shouldn't depend on its judgment — that's a harness job.

Double duty: the conversation itself was crashing the model

Lean fetches weren't the whole story. The transcript alone — reasoning, delegations, tool results — was enough to take the 12B down after a single gate. The RLM fixes that too: past a fold threshold it rewrites the conversation into a structured checkpoint. The phase artifacts on disk are the state; the transcript is disposable.

The model that had been looping uselessly delegated correctly on its very next turn. Depth kills discipline; compaction restores it.

flowchart LR
  F["fetch / search / browse
raw page ~40k tok"] --> R["RLM · Gemma e4b
resident on llama-server :8095"]
  R --> L["source-linked lead list
~600 tok"] --> A["agent context"]
  F -. "kept on disk" .-> P[".raw sidecar
provenance + citations"]
  T["conversation past fold threshold
12b ~16k · 26b ~24.5k · 31b ~28.5k"] --> R
  R --> K["structured checkpoint
goal · progress · decisions · next"] --> A
              

😑 Still not enough! Fetches were lean — but the failure log kept filling. And most of the "model failures" weren't even the model. The harness was lying to it.

Harness engineering — one harness, real subagents

So I dove into the harness — a Flue app built on Pi.

Four failure classes, four structural fixes:

And the context overhead dropped again. The OSINT tool catalogue used to sit in permanent context — 11,800 tokens the orchestrator re-read on every single turn. It's now a SQLite full-text index on disk: one shell query when the model needs a tool, zero tokens when it doesn't.

flowchart LR
  U["Reporter
(gates)"] <--> O["Orchestrator
12B · tuned"]
  O -- "task" --> I["Investigator
child session"]
  O -- "task" --> F["Fact-checker
child session"]
  I -- "fetch / search / browse" --> S["Integration seams
Crawl4AI · SearXNG · dev-browser"]
  S -- "distill" --> R["RLM · e4b
own llama.cpp"]
  O -. "compaction summaries" .-> R
  O === LS["llama-server :8080
2 resident slots"]
  R === LR["llama-server :8095"]
              

😤 STILL. NOT. ENOUGH!! The plumbing was honest, the context lean — and the model WOULD NOT hold the procedure. Re-activating skills it had already read. Researching instead of delegating. Blowing straight through gates!

Model fine-tuning on Spotlight procedures

Here I learned about procedure tuning — a different animal from the capability tuning I'd always done:

• Capability tuning teaches knowledge: domain facts, formats, styles. Useless here — Gemma-4 12B already knows what a commercial registry is.
• Procedure tuning teaches behaviour: activate the methodology skill once · advance on a bare "approved" · delegate instead of researching itself · stop at the gate. Behaviours are learnable from a tiny dataset — if the data is shaped right.
• What went into teaching it:

How many orchestrator submissions to finish?

Same real Ethereum Foundation investigation · lower is better · every gate approval and guardrail-recovery turn counts. The base 12B reached 35 submissions with report repair still unfinished. Wall time for the successful RunPod runs: tuned 12B v5 35.5 min; 26B 20.5 min. The 31B duration was not captured.

The models: gemma4-12b-spotlight-orchestrator-v5-GGUF (Q4_K_M, ~7 GB — final) and its v5 LoRA adapter, trained from 14 real gold trajectories and 12 targeted correction continuations.

🤬 AND EVEN THEN IT KEPT DYING!!! At the methodology gate. Mid-investigation. EVERY. SINGLE. EVAL. I started to despair — until the server logs confessed: this time it wasn't a model capability issue…

KV-cache and serving — the invisible half of local

failed to find available cells in kv cache · failed to restore state with size ~971MB
The model was fine. The serving layer was wiping its memory on every delegation. The fix took me deeper into how the model is served than I ever planned to go. A couple dozen iterations later, this is the serving recipe I landed on — learning what each of these levers even was along the way:

API tiers are exempt: the harness detects the tier and serves frontier models unrestricted. The constraints live exactly where the constraint exists — on the device.

🚀🥲 It ran! It was fast! But it still wasn't sovereign — every fetch and every search went through a paid API key. One last dependency to cut.

Cutting the last commercial dependency

Search and scraping ran on Firecrawl — a paid, closed API on the critical path of a tool meant to run locally and privately.

• The swap: fetch → Crawl4AI (Apache-2.0, local Playwright browser) · search → SearXNG (self-hosted metasearch). Firecrawl demoted to optional fallback.
• The surprise: the gap didn't just close. On our 8 hardest registry and civic-monitoring cases, the open stack leads.

Scraping success · 8 hostile registry/civic cases · benchmarks.buriedsignals.com

(Firecrawl's miss is instructive: on one municipal homepage its main-content filter stripped the entire page down to image markdown.)

Search tells the same story. SearXNG — self-hosted, no key, no per-query bill — edges out the commercial APIs on source quality:

Search source-quality · 8 investigative queries, scored on reaching canonical sources · benchmarks.buriedsignals.com

And then… 🎉 SUCCESS!!!