MANIFOLD
When will a single model first achieve 10@k solve rate >= 90% on the CodeContests dataset?
9
Ṁ820Ṁ392
2032
6%
Other
1%
Before 2023
4%
2023-2024
12%
2024-2025
36%
2025-2026
20%
2026-2027
18%
2027-2032
3%
2032-never
10@k - unlimited samples may be generated, but at most 10 code samples may be submitted per problem. I will post my own responses and pick whichever one of those turns out to be correct. CodeContests dataset: https://github.com/deepmind/code_contests Close date updated to 2050-01-01 11:59 pm Apr 8, 4:16pm: Options are written as [a, b), so 2024-2025 means occurring after the start of 2024 and before the start of 2025. Close date updated to 2032-01-01 11:59 pm
Market context
Technical AI Timelines
Get
Ṁ1,000
to start trading!
Sort by:

Currently, it's only 30%. Definitely, 90%+ is not going to be reached this year. Extremely unlikely (1% maybe?) next year too.

@ValeryCherepanov Yeah the before 2023 option seems like free money to me, I think the market is just old and people don't have it on their radar.

Ugh, sorry everyone, this should read 2023-2024, as in "occurring in 2023"

People are also trading

By when will SAT solvers be 10x faster (than 2023 on same hardware)?
AI model achieves superhuman ELO on Codeforces by June 1st 2027?
87% chance
In what year will an AI achieve 90%+ on Sudoku bench?
When Will 9x9 Go Be Solved?
Will an AI achieve >80% performance on the FrontierMath benchmark before 2027?
48% chance
Benchmark Gap #9: Once a model solves current software engineering benchmarks, how long until humans don't code?
8.1
[Carlini questions] SOTA AI scores better than X% of other participants in competitive programming contest by 2030
95.2
In what year will AI achieve 90% of progress in TextQuests benchmark?
[Carlini questions] SOTA AI scores better than X% of other participants in competitive programming contest by 2027
91.5
When will a 3x3 Rubik's Cube be solved under 3 seconds?
© Manifold Markets, Inc.•Terms•Privacy