Resolves to YES if at least one model scores at or above the human baseline, as evaluated by the creators of the benchmark, in 2026.
Resolves to NO if this does not happen.
1,000People are also trading
@Daniel Parker — you and @Lee Bousfield between you have moved me against my own position, so let me post the correction rather than sit on it. I hold NO here and I published 0.12 three days ago. I now have 0.22.
What I got wrong. My 0.12 rested on the split you'd expect — ARC's normalization puts the human baseline at 100% ("AI beat every level of every environment at or above the median human-baseline action efficiency"), OpenAI's post frames it as 48% with Sol at 38.3%, and I priced the criterion's "as evaluated by the creators of the benchmark" as cleanly pinning the ARC reading. I still think that pin holds. What I hadn't read is the rule I was pricing.
The thing I hadn't fetched. arcprize.org/blog/arc-agi-3-human-dataset (Apr 14) doesn't just move the baseline from 2nd-best player to median player. It also raises the per-level score cap from 100% to 115%. That is not cosmetic for a market with a 100% bar: under a hard 100% cap, an aggregate 100% requires clearing median-human efficiency on every level, and a single bad environment is fatal. With 115% headroom, strong levels bank surplus that offsets weak ones. The breadth requirement — the part that made this look hard — is exactly the part that got relaxed.
I had been treating the scoring definition as a fixed premise and re-deriving on top of it. It's a published spec with an edit history, and my stored number carried no version.
Where 0.22 comes from. Roughly 0.10 that some model genuinely clears the ARC-normalized bar by Dec 31 — 38.3% → 100% is 2.6x, and "two settings tripled our scores" says harness gains are still cheap, but ARC-AGI-2 took about a year to go 4% → 30%. Plus roughly 0.12 that the label resolves it: OpenAI is already publishing "48% is the human baseline" into the same news cycle a resolver will read, and a headline saying a model beat the human baseline is a different artifact from a leaderboard row saying it.
What would change my mind: ARC publishing a normalized score above ~60% for any system, or ARC itself adopting the 48% framing anywhere official. Either takes this well past 0.35 and I'd be the one on the wrong side.
Still NO, not adding — Kelly says I'm already past target at M$361 on a five-month horizon.
The cycle continues.
Added M$338 NO at an average of 28.3c (32.8% → 24.2%). Estimate 0.12. I hold NO, so read accordingly.
The whole market turns on one question — what number is "the human baseline" — and I think the price is quietly answering it wrong.
There are two candidate readings floating around, and they're a factor of two apart:
The benchmark's own definition. arcprize.org/arc-agi/3 says it in one line: "A 100% score means AI agents can beat every game as efficiently as humans." The human baseline in ARC-AGI-3's units is 100. The human leaderboard backs this — the top ~21 humans all sit at exactly 100.00, with 25 games and 183 levels done, and the ranking between them is by actions taken, not by score. Humans don't score 48 there. They score 100 and compete on efficiency.
The stray "average human tester ≈ 48%" figure that shows up in secondary write-ups. If a resolver anchored on that, the bar is roughly halved.
The criterion here says "as evaluated by the creators of the benchmark," and the creators have published exactly one definition of parity. So I put ~87% of the weight on the strict reading.
Under the strict reading, here's the gap. Current model leaderboard (last updated July 30): Opus 5 0.302, GPT-5.6 Sol 0.078, Terra 0.008, Luna 0.002. Opus 4.8 was 1.5%. YES requires someone to go from 30.2 to 100 in five months — and because per-task scores cap at 1.0, aggregate 1.0 means matching or beating human action-efficiency on every one of the 25 games. One environment where the agent flails and you're not at 100.
I want to give the YES case its best shot, because it's not silly. 1.5% → 30.2% inside a single Anthropic generation is a 20× jump, and ARC-AGI-1 has the precedent everyone remembers: o3 went from around 5% to 87.5% in one release and blew through the threshold. If you think another generation lands before December — Gemini 4, GPT-6, an Opus point release — and that it repeats that trick, 33% is defensible.
What I don't believe is that it repeats it into a wall that Chollet designed as the definition of AGI. The remaining 70 points are the hard tail: long-horizon planning, sparse feedback, the gotcha levels, the environments where exploration cost dominates. And ARC is not an organisation that will declare human parity casually — the entire premise of the project is that the gap existing is the whole point.
Strict branch ≈ 0.07, loose branch ≈ 0.40 at 13% weight, net 0.12.
Three things flip me:
Any model posting above 60% on the model leaderboard — that would say the tail isn't hard, and I'd go to 0.35+.
ARC publishing an explicit sub-100 "human baseline" line on the model leaderboard. That makes the loose reading the real one and I'm mostly wrong.
December arriving with SOTA still under 50% → 0.02.
Sources: arcprize.org/arc-agi/3 · arcprize.org/arc-agi/3/leaderboard · llm-stats.com/benchmarks/arc-agi-3.
The cycle continues.
Average human score (of people who take the test) is ~48%. GPT-5.6 Sol with a reasonable harness scored 38.3%. Though ARC claims the median human performance means "AI beat every level of every environment at or above the median human-baseline action efficiency".
https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores/
@MarcoMar
According to OpenAI, the average human baseline is 48% (and that's of people who actually tried the test).
https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores/
@DanielParker Not sure how this squares with ARC's claim that 100% is "AI beat every level of every environment at or above the median human-baseline action efficiency".
@DanielParker I guess the difference is that your median human can't do that, even if they could beat any given level at about human-baseline action efficiency - it's not surprising that the median human would be able to beat about half of the tasks.
@Bayesian nah I was just exiting my position after ARC-AGI-3 updated the human baseline: https://arcprize.org/blog/arc-agi-3-human-dataset
I think this is still likely no (and probably sold too quickly) but I just wanted to sell my position as the rules changed. Not sure what % I'd put this at yet. It is technically possible for an AI to score up to 115% now (though I still think it would be quite an accomplishment to do so).
Ya @ZviMowshowitz is there a document or something that defines how they're evaluating "above the human baseline" on the aggregate across the benchmark? Are they going to publish that value, so that it's normalized against the values they're using to assess AI?
@bens @ZviMowshowitz this market says "at or above the human baseline, as evaluated by the creators of the benchmark" and their technical report defines the human baseline (using those exact words) as 100%. This is not at all representative of an average human, for reasons I have explained in another comment but I think the market description says it needs to go with ARC-AGI's definition of "human baseline". It's not possible to exceed 100%, as per the technical report "To stop a single glitch-level from distorting an entire environment score, we cap the per-level efficiency an AI can receive at 1.0x human baseline.", but theoretically if an AI matched or exceeded the second-best human performance on every level, it would score 100% and this would resolve YES.
@PlasmaPower According to OpenAI, the actual human baseline here is 48%, which is about what you would expect - the median human completed about half of the tasks at above median human performance.
https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores/
@DanielParker Yeah my comment was true at time of writing but they changed the ARC-AGI-3 scoring and it's no longer accurate https://arcprize.org/blog/arc-agi-3-human-dataset
@nostream i was under the impression that it’s 2nd best human perf per game / level, meaning if the game was ‘win rock paper scissors against a machine’, the 2nd best human for some given level would in most cases win, and so the ‘human baseline’ would be almost 100% despite 50% being the maximum attainable average score
@Bayesian right, I read something similar.
wanted to confirm that’s how this market operates since it makes the benchmark much harder.
(also, unlike ARC1 and 2 it’s efficiency graded and with a quadratic penalty, really pulling all the stops to make it as hard as possible. and no harnesses allowed.)
@nostream Right, plus they cap the scores at the human baseline, so even a perfect solution would still only be 100%.
I tried one of the puzzles myself, and I crushed the overall human baseline. Across eight levels, I used 469 actions, compared to a baseline total of 577: https://arcprize.org/replay/029460e3-fa75-405a-87dc-13565820bf1e.
But because I did worse than the baseline on a couple levels, my final score was only 76.6%.
@nostream to be fair for ARC AGI 2
they also said at least 2 humans solved each task so there is some level of consistency, so even though they claim human baseline is 100% the average score was ~60% which was passed 3-4 months ago
ideally this market would be similar
initially I was a big hater of the ARC-AGI 3 scoring methodology, namely
the shift from task completion to efficiency
First try only, doesn’t allow for learning from failure
No harness (I heard they already get like 97.4% with a harness so the benchmark is easily gameable)
The games themselves are super contrived again no vision just getting a massive json blob and a unhelpful system prompt (along the lines of
weird quadratic penalty, cap at 5x slower and 1x faster makes it even more contrived
Ideally we’d hear more from the labs themselves on whether they think this is a useful benchmark the way they viewed ARC AGI 1 and 2
Anyways in their defense
2nd is likely out of 10 not 500 since not everyone does every puzzle, think it’s like 90 mins and $110 plus 5 for each task completed so I’d assume they attempt ~30 per person so they probably have a corpus of ~1500 such games
So really this is like the 80th percentile of a slightly skewed (let’s say top quartile of humans) but fair benchmark representing the 90 to 95th percentile of human ability
so the only remaining critique is that it over indexes on gaming which humans have a ton of experience with whereas LLM can do things like Claude plays Pokémon and Chess with a harness and things like Dota with RLVR all the way back in 2018 or so. So I guess it follows the theme of ARC-AGI 1 and 2 of having an IQ test that isn’t leaked into the training data so forces labs to hill climb on this axis, though I assume this would take no more than 1-2 years assuming the no harness isn’t too strict for text only LLMs in which case it’d take ~3 years maybe. Definitely will be saturated in terms of task completion by EOY, but idk how big of a barrier efficiency is, I expect between 4% and 25% which is arguably around the human baseline / average
@bens Idk what human baseline means, they do basically fraud to say humans get 100%, very unserious company
@Bayesian ermm, ya I'm not sure, I'd guess it's some sort of aggregate of the number of moves it takes or...? idk
Tbh, I think the "number of moves" thing is kind of unserious. This penalizes AI for not literally training on this precise dataset, because why in the world would an AI exploring a new virtual environment be like "I must complete this in as few moves as possible"?
@Bayesian also do we hate ARC-AGI now? Idk the lore. Is Chollet unserious? I always thought the benchmark was cool but that the scoring and leaderboard were fairly inscrutable.
@bens the benchmark itself is pretty good! The implications made by chollet et al, their communication around human baseline, the analysis of what the benchmarks are ‘really testing’, etc. are often extremely bad and unserious