the AI fluency test · private round
Prove you can get real work out of AI.
A 25-minute, three-round work sample. Brief a model on a messy ask. Catch the fabrications in a finished memo. Build a system and watch it run against cases you never saw. You leave holding an artifact: your submissions, the rubric they were graded against, and where the graders disagreed.
The format
Three rounds, weighted toward the least fakeable.
The Brief
A vague ask, a messy source packet, three candidate board summaries - one source is hiding something. Pick the summary the board should see and defend it from the data.
Specification · Context · Iteration
The Catch
A confident, finished memo with seeded defects. Flag what you would not sign, prove it from the sources, mind the token budget.
Verification · Cost
The System
Build the process a new hire could run on day one, under $0.15 per item - then it runs on 10 cases you never see.
Systems · Cost · Specification
The artifact
You leave with a report anyone can check.
Most of the grade is computed by plain code against hand-authored ground truth; the judgment calls go to a blind three-seat panel whose disagreements are printed, never averaged away. Every submission is readable in full beside the rubric it was graded on - and Round 3’s score comes from running your system on twenty cases you never saw, with the gap between visible and held-out accuracy printed as overfit_delta.
Bands: 4 Builds systems · 3 Trustworthy operator · 2 Productive with supervision · 1 Beginner
Band 3
Produces work you can ship without re-checking every claim.
73/100
held-out
10/10
overfit_delta
0
adversarial
3/3
The rubric is public
Every criterion and every scoring anchor is published before you start. Studying it is learning the construct, which is the point.
The submissions are readable
Anyone who doubts the number can read your Round 3 system in four minutes and form their own view.
The failure modes are printed
Hard caps land on the report by name. Panel disagreement is shown, not laundered into an average.