How many non-Transformer based models will be in the top 10 on HuggingFace Leaderboard in the 7B range by July? | Manifold

How many non-Transformer based models will be in the top 10 on HuggingFace Leaderboard in the 7B range by July?

16

1kṀ769

Jul 2

1H

6H

1D

1W

1M

ALL

39%

0

32%

1-2

24%

3-5

4%

6+

For resolution I’ll go to the HuggingFace leaderboard and select the ~7B and uncheck everything else.

I’ll refrain from participating in the market to stay neutral in case a hybrid case comes up. I’d count both Mamba and StripedHyena as non-Transformers.

Technical AI Timelines

Get

1,000

to start trading!

People are also trading

Will Transformer based architectures still be SOTA for language modelling by 2026?

Will Mistral's next model make it to the top 10 models in LLM Arena by the end of 2025?

By EOY 2025, will the model with the lowest perplexity on Common Crawl will not be based on transformers?

Which company has best Text-to-Video AI model end of 2025? (Artificial Analysis Leaderboard)

On January 1, 2027, a Transformer-like model will continue to hold the state-of-the-art position in most benchmark

When will a non-Transformer model become the top open source LLM?

Will the most capable, public multimodal model at the end of 2027 in my judgement use a transformer-like architecture?

Sort by:

https://x.com/ctnzr/status/1801050835197026696

does sliding window count as non-attention?

@HanchiSun I’d say sliding window is a type of attention. I’d consider LongFormers as a type of Transformer.

@HanchiSun Out of curiosity, would you bet differently if it was for the 3B category rather than 7B?

@KLiamSmith Good question. it is definitely harder to experiment with 7b than 3b. but even for 3b, i doubt more than 2 non-attention architecture will be better

People are also trading

Will Transformer based architectures still be SOTA for language modelling by 2026?

Will Mistral's next model make it to the top 10 models in LLM Arena by the end of 2025?

By EOY 2025, will the model with the lowest perplexity on Common Crawl will not be based on transformers?

Which company has best Text-to-Video AI model end of 2025? (Artificial Analysis Leaderboard)

On January 1, 2027, a Transformer-like model will continue to hold the state-of-the-art position in most benchmark

When will a non-Transformer model become the top open source LLM?

Will the most capable, public multimodal model at the end of 2027 in my judgement use a transformer-like architecture?

Related questions

Will Transformer based architectures still be SOTA for language modelling by 2026?

Will Mistral's next model make it to the top 10 models in LLM Arena by the end of 2025?

By EOY 2025, will the model with the lowest perplexity on Common Crawl will not be based on transformers?

Which company has best Text-to-Video AI model end of 2025? (Artificial Analysis Leaderboard)

On January 1, 2027, a Transformer-like model will continue to hold the state-of-the-art position in most benchmark

When will a non-Transformer model become the top open source LLM?

Will the most capable, public multimodal model at the end of 2027 in my judgement use a transformer-like architecture?

© Manifold Markets, Inc.•Terms•Privacy