AI progress, only when it's real.
Mode:ยทThe Threshold Report - methodology
This page states the commitments The Threshold Report is run on. The full scoring rubric, with exact weights, thresholds, and enforcement details, is maintained privately, applied identically to every story, and available on request at the publisher's discretion. This page has three layers: the whole system in one paragraph, the rules, and the edge cases.
The whole system in one paragraph
The Threshold Report tracks the progress of AI capability, and the ramifications of that progress. It asks the question: when does an AI capability cross the line, meaning the moment it stops being a demo, rumor, or roadmap, and becomes real -- something you can actually use, buy, or rely on?
Every segment of AI we cover has a running total that starts at 100 (because a lot had happened before we built this) and only moves when a verified event is logged with a primary source. Fixed rules assign every point (see below). Announcements score zero until they become real under the segment's rule, and items that matter but cannot count yet sit below "the threshold" as "not real yet" until they do.
The rules
What counts
An item scores only when all of these hold: it is verified against a primary source or independent reporting by multiple established outlets, and it is usable at the level the segment demands rather than demoed. Every scored item is also verified against its source before it counts. An item that is announced but not truly available yet sits on the public watchlist as "not real yet" with the reason stated, and moves to the record only when the evidence arrives.
The segments
Eight running totals: text, audio, video, image, agents, robots, science and augmentation, and the surrounding infrastructure including compute, memory, and power. Each starts at 100 on its start date and moves only on verified events. Totals are cumulative since day one.
How much a verified event moves its segment:
| Event | Segment effect |
|---|---|
| Milestone - a capability crosses a line for the first time | +10 |
| Notable - a clear, verified improvement to an existing capability | +5 |
| Minor - access expands, prices fall meaningfully, or a smaller verified step ships | +2 |
| Regression - something verified is withdrawn or degraded | takes back the points it earned (-3 when it predates the record) |
| Restoration - a logged regression is reversed | returns the reversed points |
| Context - real news with no capability change: incidents, partnerships, policy fights | 0, reported but never scored |
Per issue window a segment can log at most one milestone, two notables, and three minors, and can gain at most 20 points net. A busy news week earns a bounded move, never a spike.
The claims board
The board holds plain-language claims and answers each with one of three statuses.
- No: no public system does it reliably.
- Sort of: demos, narrow cases, or heavy supervision.
- Yes: ordinary users get it reliably from shipping products.
Answers change only on primary evidence and sustained availability, never on a single demo. Downward flips are allowed under the same bar, and a flip is news: it leads the next issue with its evidence linked. A newly added claim gets its first answer on the next daily update, recorded as a starting point rather than a flip.
The risk and readiness dial
The dial is a running tally of verified events that push toward control or toward chaos, cumulative since day one. Only verified items move the dial; announcements and speculation never do. Words alone never move it in either direction: an open letter, a pledge, or a call for a pause is reported as real news and moves nothing until someone acts on it - a rule takes effect, a lab verifiably changes behavior, or the signers do more than sign. Every story that moves the dial says so in its note, with the direction and the reason, and every safety story that does not move it says why not. The tally of both sides is shown next to the needle.
How far one event moves the needle:
| Impact | What it looks like | Needle movement |
|---|---|---|
| Minor | contained or single-organization impact: one exploited flaw, one deployed defense | 4 points |
| Notable | sector-level impact: real misuse at scale, a rule in force, real alignment progress | 8 points |
| Major | systemic impact: widespread exploitation in the wild, a landmark rule, a fundamental interpretability result | 15 points |
Chaos adds the points, Under control subtracts them, and the needle is the running sum, capped at 100 each way.
Independence
No scored item is ever paid for. Sponsorship buys a clearly marked slot, never a point, a status, or a placement in the record. Paid tool listings on the board carry a visible marker and a disclosure. The publication has no positions it is paid to defend.
Corrections
Errors are corrected in the open: the record entry is amended with a note, not silently rewritten, and material corrections are announced in the next issue.
Edge cases
Rumors that later confirm enter the record on the confirmation date, not the rumor date. A capability that regresses can lose points under the same evidence bar that granted them.