r/mlscaling • • Mar 31 '25

The case that AGI is coming soon

Thumbnail
80000hours.org
5 Upvotes

r/mlscaling • • Nov 22 '22

GPT-4 rumours

Thumbnail
twitter.com
0 Upvotes

r/mlscaling • • May 01 '24

So...what's "gpt2-chatbot"?

2 Upvotes

I know research papers are preferred here instead of speculation, but this could be interesting.

On April 27th, a new model debuted on lmsys. It was removed on the 30th. Here is a summary of known facts:

https://rentry.org/GPT2

Evidence points to it being a new OpenAI model, with reasoning capabilities similar to GPT4/Claude 3 Opus, perhaps exceeding them in some cases. Hard to say. I wish I'd used it more.

Here's it designing a game one-shot, and creating ASCII art (note that this exact image was on the internet, so this constitutes memorization of data, not actually drawing a unicorn from scratch).

It also had a nice writing style, with less of that ChatGPT "delve" stuff people complain about. This was impressive to see—I haven't seen an LLM with such a proficient grasp of rhyme and language.

On the other hand, it still falls for simple token-based tricks. (GPT4 got this right, Claude 3 Opus wrong. GPT3.5 and Gemini Sucktra didn't notice the word was misspelled.) And for some tasks it seems inferior to GPT4.

What do you think this is?


r/mlscaling • • Mar 26 '24

OP, Hardware, MS, OA Kyle Corbitt on GPT-6 training cluster

Post image
51 Upvotes

r/mlscaling • • Aug 01 '21

D, Hist, Forecast There haven't been any massive strides in Natural Language Processing in a while- should we be worried?

0 Upvotes

It's well over a year since GPT-3 came out, and thus far there are no obvious signs of a successor. When we look at various specific benchmarks what we see is also worrying. It's been quite some time since there's been rapid progress on a range of different NLP tasks including:

The ARC reasoning challenge: https://leaderboard.allenai.org/arc/submissions/public

RACE: http://www.qizhexie.com/data/RACE_leaderboard.html

SuperGLUE: https://super.gluebenchmark.com/leaderboard

Is it possible that there was a "golden age" of rapid NLP progress, and it's now over, or is this just a brief lull with no special significance?

I've got to say that if we have left a rapid period of special progress I find that depressing because it did feel like we were on the verge of some truly astonishing achievements for a hot moment there.


r/mlscaling • • Sep 07 '23

Forecast Sam Altman forecasts GPT-10 will be "smarter than all of humans put together"

Thumbnail
youtube.com
0 Upvotes

r/mlscaling • • Jul 29 '26

A Frozen 12B Beats Frontier Models on Verified Work: 100% Accuracy, 0 Tokens, Bit-Exact, Forever

Thumbnail
arxiv.org
3 Upvotes

r/mlscaling • • Dec 20 '24

T 7+ years of LLM highlights (2017–2024)

Post image
0 Upvotes

r/mlscaling • • Feb 01 '26

Smol, Code "Shrinking a programming-language classifier model to under 10kb", David Gilbertson 2026-01-28

Thumbnail itnext.io
0 Upvotes

r/mlscaling • • May 09 '24

Has Generative AI Already Peaked? - Computerphile

Thumbnail
youtu.be
14 Upvotes

r/mlscaling • • Jun 02 '24

N, X, Hardware xAI: 100k H100s in a few months and ~300k B200s with CX8 next summer

Post image
17 Upvotes

r/mlscaling • • Sep 13 '24

[Video] AI can't cross this line and we don't know why.

Thumbnail
youtube.com
3 Upvotes

r/mlscaling • • Sep 17 '23

Forecast What will GPT-2030 look like? (Jacob Steinhardt, 2023)

Thumbnail
bounded-regret.ghost.io
3 Upvotes

r/mlscaling • • May 04 '23

GPT-4 architecture: what we can deduce from research literature

Thumbnail
kir-gadjello.github.io
1 Upvotes

r/mlscaling • • 1d ago

N, NV, Hardware, Econ "Ex-Nvidia CEO misled top lawyer on affair with Chinese businesswoman, sources say"

Thumbnail reuters.com
15 Upvotes

r/mlscaling • • Jul 26 '25

R Potential AlphaGo Moment for Model Architecture Discovery

Thumbnail arxiv.org
0 Upvotes

r/mlscaling • • May 01 '26

N, T, A "Introducing Claude Opus 4.7"

Thumbnail
anthropic.com
0 Upvotes

r/mlscaling • • Aug 26 '23

Hist, C, Code, Data, Emp "Deep Neural Nets: 33 years ago and 33 years from now", Andrej Karpathy (time-travel experiment: implementing Le Cun 1989 on new hardware & data & NNs)

Thumbnail karpathy.github.io
0 Upvotes

r/mlscaling • • Apr 01 '22

Scaling up isn't a path to singularity

Thumbnail
mkaic.substack.com
0 Upvotes

r/mlscaling • • Nov 29 '24

N, Econ, Data, OP fascinating first-hand anecdotes from "digital sweatshop workers"

2 Upvotes

I'm not endorsing the politics here, but I think everyone in this subreddit would find this quite interesting!

https://youtu.be/qZS50KXjAX0?si=X9Rt6hVmULDbJHu2https://youtu.be/qZS50KXjAX0?si=X9Rt6hVmULDbJHu2


r/mlscaling • • May 03 '25

OP, Econ Why Open Source Will Not Win the AI Race

5 Upvotes

Open source (Either true open source or non-profit) appear to thrive in fields with low hanging, but hidden fruit. Closed source appears to thrive in fields with high hanging, but visible fruit.

AI used to fall into category 1, where the fruit was so low hanging that a non-profit like OpenAI with the right perspective, a small team, and cheap scaling could see the hidden fruit and quickly scoop up $300 billion in value.

However, now AI has entered category 2, where everyone sees the fruit but it's high up in the trees. At this point you need to be closed source and for-profit in order to brute force scale past thresholds (Regulatory, Technical, etc).

My best evidence for this is that OpenAI themselves, the open source non-profit, realized they needed to be closed source for-profit in order to win the AI Race.

*Edit Note*

One user correctly pointed out that I should have clarified by just creating a new category like Closed For Profit company. What I was trying to mean is that the winner of AI will most likely be "Closed Source" and "For Profit".

This is coming from a pattern I've observed where I don't know of any industry where there is high hanging, but visible fruit where the marketshare winner isn't closed source and for profit. For example, I don't see an Nvidia competitor that is:

(1) open source, for profit

(2) closed source, non-profit

(3) open source, non-profit.

However, the user mentioned Red Hat so I'll need to look into them further to see if the pattern I've observed still holds. However, my bet is that they are probably a newer business in an area of low hanging fruit. Where with the right perspective, a small team, and cheap scaling they can scoop up to even $300 billion in value just like OpenAI did with AI.


r/mlscaling • • Nov 13 '24

N, Hardware, X Elon Musk’s Supercomputer Freaked Out AI Rivals - TheInformation (extended snippets)

Thumbnail theinformation.com
0 Upvotes

r/mlscaling • • Feb 03 '25

First (?) serious attempt to have a language model write a journal article from scratch? "Revisiting the McKinley Tariff of 1890 through the Lens of Modern Trade Theory" by o3 Deep Research (2025)

Thumbnail kevinbryanecon.com
0 Upvotes

r/mlscaling • • Jan 25 '21

"Why general artificial intelligence will not be realized" (Nature) [D]

Thumbnail nature.com
0 Upvotes