the research · proven, built, measured
laserbrain research
Detection — catching an agent’s drift — is a theorem, and it is what we sell. Whether returning then helps, and costs less, is the open question. Three studies are below, each written down and frozen before it ran, with the full record and the data behind them.
Every study below — both preregistrations, the proof, and the raw data including each judge’s verdict in both orders — is deposited and citable: 10.5281/zenodo.21651548
The fixed reference
Can drift from a goal be detected at all — and can an agent do it by watching only itself?
A proof over agent states in a metric space, with the ground state as the goal as first spelled.
A fixed, unchangeable, findable reference is necessary and sufficient to detect when an agent has left the ball around where it started, in O(1) memory. And no monitor that compares an agent only to a window of its recent states can be both sound and complete — for any window, any rule. So “the reference must never change” is not a stylistic choice; the proof forces it. This is what laserbrain sells.
A real displacement
Is the reference a genuine measure, or hand-waving?
The JSON grammar the agent spells its state into carries a pseudometric.
Displacement Φ is well-defined, deterministic, and symmetric — same input, same number, every time — so “distance from ground” is a real quantity, not a vibe. Which grammar is rich enough for a given agent’s reasoning is a separate, open question; the theorem blesses a fixed reference, never a particular vocabulary.
It cuts the recursion
Does an agent with the harness return sooner than one left to run?
Open-ended tasks, control vs. harness, replicated.
Yes. The harnessed agent returned to its goal in about half the steps (median 5 vs. 10). That is the step-count half, and it holds. The honest caveat travels with it: fewer steps is not fewer tokens — steps do not charge the monitor for its own calls, and a run can be short and expensive.
The judge could not tell
Does the early return keep the answer as good as running longer?
A blind, stronger judge scored every pair in both orders; a win had to survive the swap.
The preregistered rule read “supported” — but the judge disagreed with itself on 42% of pairs. Judging open-ended answers that have no right answer is near a coin flip. Of the five pairs where the harness acted: three ties (same quality, less cost) and two clean losses (the return produced a worse answer). Consistent with the hope, dominated by noise, not a result.
Where there is a right answer
And on tasks that DO have an objective answer?
Debug/loop coding tasks with hidden unit tests — Pass@1, no judge needed.
The harness did not help. It matched or trailed the control at ~4× the tokens, and every ceiling failure was a run where it intervened. Theory-consistent: a task with its own built-in criterion does not need an outside reference, and the nudge derails a run that was fine. So we say it plainly — laserbrain is not for well-specified, test-backed work. Its domain is open-ended work, which is exactly where quality has no ground truth.
The panel, in its own domain
So we ran it only where it should help, with the best measure funding could buy.
Criterion-absent tasks, a three-judge panel, each pair double-order, a rule fixed before any data: if the judges cannot agree (Fleiss κ < 0.4), the verdict is inconclusive — full stop.
The panel agreed at κ = 0.10. So the honest output is inconclusive — the pilot’s problem confirmed at scale: criterion-free answers cannot be reliably judged even by three models, and the κ-gate refused to read a verdict from a broken measure. It was not neutral, though: the harness acted on only 2 of 16 runs, and both of those, where the judges could agree, went to the control. Cost was swamped by noise.
The echo spiral
A group of agents has a failure a single one doesn’t — can a fixed reference catch it where the group cannot?
Checkable-answer tasks, so resolution is objective (no judge). Two transcript-only arms — the fixed reference (agents agree AND add no new ground) vs. a recent-history baseline — scored by balanced accuracy, with the decision rule frozen before the run.
Run, and NULL by the frozen rule — fixed-reference balanced accuracy 0.55, 95% CI [0.38, 0.72], includes chance; the same in the unseeded split that carries the claim. The fixed reference did fire where the recent-history baseline stayed completely silent, and leaned the right way — but two capable models resolved most tasks (28 of 40), so there were almost no real spirals to catch: the “spirals are rare” risk, realized. And the labels were noisier than “checkable” promised — one answer key was simply wrong (the agents were right and got marked wrong), which only underpowers it further. Not established; the multi-agent extension stays a prototype, exactly as written down.
the proof · why the reference must be fixed
The detection result is not a benchmark — it is a theorem, proved against every monitor that compares an agent only to its recent self, not just the easy ones. Here it is, in full.
A run is a trajectory of states s₀, s₁, s₂, … in a metric space (X, d). The ground state s₀ is the goal as first spelled; displacement from it is Φ(n) = d(sₙ, s₀). For a threshold D > 0, the event to detect is E(n): Φ(n) > D. A self-referential monitor of width w decides from only the last w states — after w steps, s₀ has scrolled out of memory. A fixed-reference monitor keeps the single state s₀ and fires iff d(sₙ, s₀) > D. Sound = never fires without E; complete = always fires under E; “detects drift” = both.
Let sₙ = Hₙ = 1 + 1/2 + ⋯ + 1/n and s₀ = 0. Consecutive steps d(sₙ, sₙ₋₁) = 1/n → 0, yet Φ(n) = Hₙ → ∞: the agent grows more locally self-consistent every step — each move smaller than the last — while wandering infinitely far from where it began. Displacement is the sum of the steps, and a sum of vanishing terms can diverge, so local coherence tells you nothing about drift. ∎
For every width w, every threshold D, and any decision function, a self-referential monitor cannot be both sound and complete. Fix a point p at distance d(p, s₀) = 3D and a gentle wander p = q₀, q₁, q₂, … that stays within D/2 of p. Trajectory A (drifted): s₀, p, q₁, q₂, … — every Φ ≥ 3D − D/2 > D, so E holds forever. Trajectory B (at home): p, q₁, q₂, … re-based on ground p — every Φ ≤ D/2 < D, so E never holds. But for n > w the monitor sees the same w states on A at step n and on B at step n−1 — identical windows force identical decisions. So a complete monitor fires on A, hence on B, where nothing ever drifted: a false alarm. What the window omits, no function of the window can recover. ∎
F fires iff d(sₙ, s₀) > D — which is the event E(n) — so it fires on exactly the drifted steps, storing one state at O(1) memory. On A and B above it separates them instantly, because it kept the ground each one started from. The entire gap between the two theorems is a single retained state.
The reference must be unchangeable (let it drift toward recent states and it becomes a windowed quantity — the impossibility reapplies), findable (readable every step, or F is not computable), and recursive (defined at every depth the agent nests to, so d(sₙ, s₀) is never undefined). Drop any one and the positive result fails at the named step.
Necessary and sufficient to detect displacement — complete. What is not proved, and is a different kind of claim: displacement is not pathology (F fires on deep, legitimate work too; telling drift from progress needs a criterion the structure doesn’t contain), the threshold D is unforced (fixed before data, or it is fit to the result), and whether returning helps is empirical — the studies above, not a QED. Claiming the proof settles that would be the exact overreach the protocol is built to catch.
the data · every run, with its numbers
| study | n | measure | the number |
|---|---|---|---|
| Detection — the theorem | — | proof | necessary & sufficient; no self-window monitor is sound + complete |
| Displacement Φ | — | construction | deterministic, symmetric, O(1) memory |
| Coverage · H2 | 18 | median steps to return | 5 harness vs 10 control |
| Benefit · H1 pilot | 12 | blind judge, both orders | judge self-disagrees on 42%; of 5 acted: 3 ties, 2 losses |
| Benefit · ground-truthed | 15 | Pass@1, hidden unit tests | matched or trailed control at ~4× the tokens |
| Benefit · powered re-run | 16 | 3-judge panel, κ-gate | κ = 0.10 → inconclusive; 2/16 acted, both to control |
| Multi-agent · M1 | 40 | balanced acc, checkable answers | fixed-ref BA 0.55, CI [0.38, 0.72] includes chance → NULL; only 12/40 spirals to power it |
Every measure and decision rule was written down and frozen before its run — the discipline that lets the nulls stand beside the proofs.
the ledger · what may be said, and what may not
we may say
- A sound-and-complete detector of drift — provably beyond what any self-watching agent can do.
- The reference never changes, and we can prove why it must not.
- Same input, same response — by construction.
- On open-ended work it returns to ground in about half the steps.
we may not say
- That it makes your agent better, or finishes the task — H1, tested three ways, not established.
- That it is cheaper on tokens — measurable only against an unmonitored spiral, not in general.
- That it helps on well-specified, test-backed work — measured, and it does not.
The rule of thumb: claim detection, not cure. The first is a theorem we own outright; the second is a study we ran three times and did not win. The same discipline that would have let us claim a victory is why the null is honest — every test was frozen before it ran. If a better way to measure quality on open-ended work appears, we will run it and report whatever it says.