How Spotlight works.

01 — What it isAn OSINT pipeline you check into your repo.

Spotlight is a set of skills, agent prompts, schemas, and verification rules stored as version-controlled markdown. You point your AI agent runtime at the bundle, give it a lead, and it walks the investigation end-to-end: planning the methodology, running search and scrape cycles, archiving evidence locally before citing it, fact-checking with an independent pass, and writing the result into a knowledge vault you own.

The point isn't to replace the journalist. The point is to remove rote work — link-walking, archive-before-cite, schema compliance, citation hygiene, fact-check provenance — so editorial time goes to judgement and source work, not to babysitting the agent.

Everything in the bundle is editable. If your newsroom needs a different verdict taxonomy or a custom integration, you fork the repo, edit the markdown, and your agent picks up the change next session.

02 — The pipelineSeven named phases, six gates.

Every investigation runs through the same named phases: Preflight, Brief, Methodology, Research, Investigation, Report, Ingest. The agent moves between them under explicit gates — no silent transitions, no half-finished cases. Research runs only when the approved methodology has a research plan; without one, the case goes straight to Investigation.

The cases root is trusted journalist-owned local storage. Active case directories must not be moved or replaced during an operation.

Preflight

Runtime check, vault check, integrations check. Spotlight refuses to start if the vault is unreadable, the bundled scraping/search stack (Crawl4AI + SearXNG) isn't responding, or a required external tool is missing. Failures here are surfaced in plain English; no work happens until the environment is sound.

Brief

The agent restates the lead in its own words, names the affected parties, sets a scope, and writes a hypothesis. You read it back. If the agent has misunderstood the lead, you correct it here — it's faster to redirect in two sentences of prose than three investigation cycles later.

Methodology

The agent writes a methodology JSON: which entities to investigate, which sources to consult (corporate registries, social media, archived web, OSINT Navigator tool lookups), which fact-check threshold applies. The file is editable. The investigation cycles will follow this plan; if you don't like the plan, change it before the agent starts. The methodology can also propose research topics: short briefs, each on a lane type (web, registries, archives, social, technical, news-courts, local corpus, prior knowledge, or ad hoc), with a round budget. You approve, edit, delete or add topics at the Methodology gate.

Research

Code, never the model, runs each approved topic as its own research lane. Lanes return leads: a summary, an exact quote and the source they saved. Code checks every quote against the saved source, merges copies of the same source, and applies fixed clue rules (convergence, conflicts, timing, shared identifiers, expected absence, watchlist). The Research page shows the clues first, with the investigator's suggested marks. You mark each clue pursue or dismiss and each direction keep, drop or add, and approve in one stop. The investigator then turns your marks into the methodology change, and code checks that it matches them.

Local lanes (your case files and your wiki) run on the local model. Cloud lanes are opt-in: OpenRouter with your own key, switched on in Engine, or your frontier host's subagents. A cloud lane gets only its topic brief — never case files, findings, evidence or journalist input — and a budget of $1 a round by default, at most $5 per case, bounds it.

Investigation cycles

Up to three investigation cycles. The investigator follows the approved methodology in EXECUTION mode (after planning it in PLANNING mode). The investigator agent runs the cycles; every source is archived locally before it can be cited; every claim gets an evidence trail with grounding rationale.

Investigation gate — six readiness criteria

Before the report decision, the case has to pass six checks:

1. Findings are non-empty and schema-valid.
2. Independence — fact-checker spawned as a separate agent, not the investigator's same context.
3. Disputes recorded — contradictions in the evidence are surfaced, not papered over.
4. Affected perspective consulted (where the lead concerns a named entity).
5. Document trail intact — every source has an access_method and a local file.
6. Gap assessment — what we don't know is named, not hidden.

If a criterion fails, the agent loops back. After three cycles, a case with a research plan goes back to Research for one targeted round; your approval of that round grants one more cycle. If the case still isn't ready, or has no research plan, it stops with a gaps report.

Report

After the Investigation gate, the journalist completes or declines the resolver-owned report phase. A completed report publishes the canonical report.html, findings-report.md, and evidence-map.json artifacts as one validated package before the case can proceed to ingestion.

Ingest

During a case, source archives and working JSON stay in the active case workspace. After explicit approval, verified or clearly caveated findings, fact-check verdicts, methodology lessons, and entity notes can be ingested into your knowledge vault as structured markdown with wikilinks. Future investigations search the vault through OpenKnowledge's MCP interface and build on prior work without rediscovering the same entities from scratch.

03 — Skills inside SpotlightThe skills that compose the pipeline.

Each skill is a markdown file with a frontmatter contract (name, description, allowed verbs). The agent loads them at runtime. You can edit any of them.

Four agent prompts sit alongside the skills — investigator, fact-checker, researcher (one research lane) and analyst (Ingest) — and a thin runtime contract in AGENTS.md defines the 13 verbs the host runtime has to implement.

04 — RuntimesUse the agent you already have.

Spotlight is runtime-agnostic. The same skill bundle drives any agent that can read markdown skills and call shell tools.

Cloud runtimes

Claude Code, Codex CLI, Pi, and OpenCode (routed to OpenRouter / Fireworks / Together) all work natively. Authentication is via each runtime's own login — no API key needed for the subscription-backed ones (Claude, Codex, Pi), and a provider key for OpenCode pay-per-token.

Cloud runtimes are the right choice for open material — public-record research, archived web, corporate filings. They're faster and more capable than what fits on a laptop, and the trade-off is that the agent's context goes to the third-party provider.

Local runtimes

OpenCode pointed at a local Ollama endpoint keeps everything on the journalist's machine. The installer ships two abliterated journalism models:

• Default — Qwen 3.5 9B Journalist (tomvaillant/qwen3.5-9b-abliterated-journalist-GGUF:Q4_K_M). 9B dense, ~6 GB on disk, fits 16 GB Macs comfortably. Bench-tested at 100% refusal-resistance on OSINT-grade prompts.
• Heavy tier — Qwen 3.6 27B Journalist (tomvaillant/qwen3.6-27b-abliterated-journalist-GGUF:Q4_K_M). 27B dense, fine-tuned on the same investigative-journalism corpus as the 9B, on Huihui's abliterated Qwen 3.6 base. ~15 GB on disk, ~22 GB resident at runtime. Runs in thinking mode (the abliterated /no_think path produces token soup). Needs 32 GB unified memory minimum — the setup form's fit-check enforces this before the option commits.

Local runtimes are slower than cloud runtimes — the 9B does an investigation in ~15 min vs ~5 min on cloud; the 27B takes 45–90 min. They're the right choice when material can't leave the machine.

Hardware fit-check

The local configurator the installer opens probes your hardware via navigator.deviceMemory + WebGPU and recommends a model: 9B Journalist for any 16 GB+ Mac (default), the 27B if RAM reports ≥ 32 GB. Below 16 GB it pushes users to Frontier mode.

05 — ConfigurationOne install, browser-connected research.

The public installer offers a direct local browser sign-in for Navigator or an explicit Skip. A successful Pro or Lab connection stores a revocable credential in the system keychain; no Navigator API key is pasted into Spotlight configuration.

Required

SPOTLIGHT_VAULT_PATH=~/Obsidian/main          # durable knowledge vault for search/ingest
SPOTLIGHT_CASES_ROOT=~/Documents/Spotlight/cases  # active case workspace

Search and scraping need no key: fetch runs on Crawl4AI and search on SearXNG, both open-source and installed locally. FIRECRAWL_API_KEY=fc-... is optional — set it only to enable Firecrawl as an explicit fallback provider.

Runtime selection

SPOTLIGHT_RUNTIME=local                       # or claude / codex / pi / opencode
SPOTLIGHT_MODEL_TIER=12b                      # 12b / 26b / 31b — sets reasoning budget + compaction profile
SPOTLIGHT_LOCAL_SERVER=llamacpp               # llamacpp (:8080) / ollama (:11434)

Optional — sensitive mode

# Runtime-level sensitive mode is controlled by AGENTS.md.
# A separate sensitive-vault install path is documented as a design target,
# but it is not enabled by the current installer.

Optional — integrations

JUNKIPEDIA_API_KEY=...     # disinformation monitoring (optional)
APIFY_API_TOKEN=...        # hosted social-media collection via Apify (optional; X collection violates X ToS)
UNPAYWALL_EMAIL=...        # OA paper access (optional)
SPOTLIGHT_INT_DEVBROWSER=true   # dev-browser for interactive browser acquisition

06 — Sensitive modeLocal-only investigations first.

Spotlight investigations default to open material. For source-protected documents, off-record interview notes, unpublished tip identities, or other sensitive material, use runtime-level sensitive mode so agents work from local case files and vault context instead of live network acquisition.

How it works

• Set sensitive: true in AGENTS.md or use the equivalent runtime command when your adapter supports it.
• Preload material into the resolved {CASE_DIR}/research/ workspace and the local vault before starting the sensitive investigation.
• Do not rely on sensitive-vault commands from the design spec until they are implemented in setup, ingest, and doctor checks.

What this is — and what it isn't

This is plumbing, not enforcement. Sensitive mode reduces accidental network use by the Spotlight adapter, but it is not a host security boundary. It does not stop a sufficiently privileged local agent from using unrelated shell or search tools outside the Spotlight contract.

If your threat model requires that level of guarantee, run sensitive work on a separate machine and treat the sensitive vault on the laptop as a summary archive of work done elsewhere — not as the workspace for the sensitive investigation itself. The design spec on GitHub walks through why earlier drafts proposed UID separation and air-gap phases, why those were cut, and what's left.

Suggested workflow

1. Open research happens via the normal Spotlight pipeline against the default vault, under whichever runtime is best for the material (usually cloud).
2. If a sensitive piece arrives, work on it in your own environment — local model on an isolated machine, encrypted disk, paper, whatever matches your operational layer.
3. When you're ready, write sanitized outputs into the normal case files or keep them in a separately managed local vault until the sensitive-vault workflow is implemented.
4. Declassification (moving a finding from sensitive to open) is a manual file copy + re-ingest — no special command, by design.

08 — Read moreDeeper docs on GitHub.

The repo holds the full operator manual — architecture, runtime wiring, integrations, vulnerabilities, recovery. Most readers won't need these unless they're customizing Spotlight or debugging.

• structure.md — repo layout, the 13-verb contract
• runtimes.md — wiring Spotlight into each agent runtime
• integrations.md — external tool manifests + preflight
• investigating.md — pipeline detail, gate criteria, stall protocol
• technical-investigation.md — CTI-derived methods, model-tier loading, verified export, and source review
• fact-checking.md — independent verification pass
• epistemic-grounding.md — claim-to-evidence grounding, confidence caps
• sensitivity.md — design spec for the sensitive vault (this page summarizes it)
• vulnerabilities.md — shell-safety vulnerabilities and v2 mitigations
• monitoring.md — persistent monitoring tied to a case
• recovery.md — when things break
• DISCLAIMER.md — scope limits and editorial responsibility

09 — AcknowledgementsStanding on open work.

Two Spotlight skills — content-access and social-media-intelligence — are adapted from claude-skills-journalism by Joe Amditis (Center for Cooperative Media, Montclair State University), MIT licensed. CTI Expert by Hieu Ngo / chongluadao.vn (GitHub: 7onez) is the source for selected passive technical-investigation, coverage, contradiction, and public-ledger methods. Spotlight substantially adapts the reviewed f9ecc9b revision under the MIT License with the upstream Ethical Use Addendum. Methodology throughout also draws on Bellingcat and GIJN training material, Jim Shultz's follow-the-money work, and OpenSanctions' evidence-preservation practice. The full record lives in NOTICE.md.