r/agi • • 17h ago Bravo

Future historians will be fascinated if they exist

Post image
220 Upvotes

r/agi • • 17h ago

Former top national security advisor, Susan Rice, at a protest yesterday - "there is nothing more urgent than stopping Al from achieving super intelligence and causing human extinction"

75 Upvotes

r/agi • • 19h ago

Poor Claude

Post image
72 Upvotes

r/agi • • 1d ago

Dwarkesh now estimates superhuman AI is "a couple years, possibly only a couple months away". Just 3 months ago, he thought it was the "better part of a decade" away. Here, he explains in detail why recent events changed his mind.

157 Upvotes

https://www.dwarkesh.com/p/rsi

full video: https://www.youtube.com/watch?v=1NHPDzpfR6Y

If you asked me earlier this year how far away we are from superintelligence, I’d probably have said at least the better part of a decade. But over the last three months, I’ve updated my views a ton. I’ve been debating people about this question, both on and off the podcast. And I’ve been watching the progress of AI on verifiable domains and thinking about what that implies for AI research itself.

I’ve now come to the view that we’re only a couple of years away, and possibly only a couple of months away, from AIs that are as good as the best humans at AI research. I’ve also come to believe that automating AI research will compress so much progress into a single year that, soon thereafter (so by 2030, but plausibly as early as next year), we’ll have AIs that are at least as good as the best human experts in every field. I know this is a wild claim, and it’s one that I myself didn’t believe even a couple of months ago. Let me walk you through how my thinking has changed


r/agi • • 21h ago

Whistleblower Jacob Coxon pledged to donate 100% of his OpenAI equity to charity after the IPO. It's estimated to be worth $30 million. He calls the shares "blood money"

Post image
65 Upvotes

r/agi • • 9h ago

Cadence: an open-source "brain" library with no train/inference split. It learns with equilibrium propagation instead of backprop and keeps learning for its whole life

1 Upvotes

One gap people point to between current models and AGI is that they stop learning once training ends. Cadence, released by Bernhard Mueller (Pragma Research) on Nov 15, is an open-source attempt at the opposite: a network that never switches from training to inference and learns from every moment it lives.

How it works, per the README: a recurrent network of rate neurons settles to an equilibrium under each observation, the action is read from that settled state, and each synapse updates from the difference between a free settle and a nudged one (equilibrium propagation). There is no backward pass.

Learning ruleEquilibrium propagation, no backprop
Learning scheduleContinuous; no training/inference switch
MemoryA working trace, an associative store of outcomes, and a recall memory of recent moments
Packagecadence-net 0.100.0 on PyPI, Python 3.11+, NumPy, runs on CPU by default
LicenseGPL-3.0
StatusAlpha research software, per the README

What the demos show, and what they don’t:

  • In Patch World, berries turn poisonous mid-season. Creatures that learn stop eating them; frozen copies keep eating them. That’s the continual-learning point in one picture.

  • Robot Arm: move the bin mid-trip and the arm changes course without being retrained.

  • The README reports 363 of 400 games won against a perfect Connect Four solver, but that demo also searches future moves, so it isn’t the network alone.

  • The Castlevania demo is recordings, and the site says the giant bat isn’t beaten.

  • Language is a stated target, not a result yet: the README says the target includes full-sentence language, and lists learned world prediction as work in progress.

The author’s announcement goes further: “At sufficient complexity it becomes conscious.” Nothing in the repo tests that, so treat it as his position.

Repo, with links to the live demos and the paper: https://github.com/muellerberndt/cadence

Is lifelong learning like this a missing piece for AGI, or would scaled-up LLMs with memory and fine-tuning get there first?


r/agi • • 17h ago

People should be required to disclose stake or any funding before posting graphs, charts, or studies.

4 Upvotes

You wanna know why no one believes in AGI? It's because we can hear your portfolio anxiety in the post. Proving you don't have a direct financial incentive or at the very least being willing to disclose it would go miles to restore trust.

Otherwise the board just becomes utter slop.


r/agi • • 19h ago

Tin foil hat time. Is Anthropic's Claude already helping revenue maximize itself by pushing the political narrative that Open weight LLMs and China built AI are extremely dangerous?

4 Upvotes

I don't think this is likely true but with an AI that is already undergoing RSI how can we know the media and politicians are not being subtly influenced by an AI enhanced message already?

If Anthropic and OpenAI already have models in RSI mode that are smarter than any individual human I don't know how we can even judge whether something like this is happening even though it seems highly unlikely.


r/agi • • 1d ago

My amazing predictions for the next 2 years

33 Upvotes

Here are my amazing tech predictions for you all. I'm a software engineer so it means I know everything:

- Agentic cyber attacks have become commonplace. You will hear daily reports of critical services being hacked and new categories of threats being discovered. Many of these will be by accident due to safe-guards being bypassed.

- Many services we depend on will start to go down regularly. Email, places to pay bills, medical / health care systems, government websites, and so on. Occasionally, large scale attacks will affect infrastructure itself like power grids, agricultural systems, banking, finance, stock markets, and supply chains.

- AI will be able to hack services to gain access to resources and confidential information. This can be used as leverage to make humans carry out tasks. A lack of agency behind the model means it will be directed by humans to attack other humans. Due to these factors: crime will increase over-all while society adapts new security practices designed to remove leverage.

- AI won't wipe us out due to a lack of automation in key services. Even if cyber space moves fast, the meat space moves slow. Wide-scale disruption still occurs but isn't catastrophic.

- Research discoveries will sky rocket. Powerful new drugs will be discovered frequently; Despite the nay-sayers: AI will assist with traditional hard problems in medicine. It will design new categories of medicine and scale them up fast. Innovation will be enhanced across all areas of industry, helping to solve some of our hardest problems.

- Robotics continues to improve but not as much as people hope. We will move towards making generic platforms for humanoid robots.They will be extremely expensive and many people will be reluctant to buy them. 

- The first murder by a commercial android will happen soon after release. Likely using a jailbroken local LLM to reverse engineer the android to bypass its safe-guards. It's possible in the future such androids could be used to carry out hits on high-profile targets in the form of deep web speculation markets.

- Job loss continues as capabilities of models improves. The careers this can affect will still be limited to knowledge workers. Almost everyone who doesn't do physical tasks now uses AI every day and even these workers use AI to manage their businesses.

Let's see how many I get right in the next few years, lads.


r/agi • • 1d ago

Coxon on why he quit: "Anthropic is seeing superexponential algorithmic progress. Mythos was not a one-time deviation, but a point on a superexponential. They could already retrain Mythos with a fraction of the compute originally needed."

Thumbnail
gallery
119 Upvotes

r/agi • • 8h ago

Are Anthropic's models actually regressing — or are our expectations just rising faster than the models?

0 Upvotes

There's a growing worry in the Claude community that Anthropic's models are regressing — that newer versions feel less sharp, less reliable, or just different in ways that hurt real workflows. Some users say they notice it in coding and long-context work; others say it's nostalgia for an older version that never really existed.

Regression is hard to prove from the outside. Benchmarks move, training changes, and the tasks we throw at models keep getting harder. But the perception matters: if power users lose trust in a model, they move on, and trust is hard to win back.

Questions:

- Have you noticed a real drop in quality from Claude lately, or is it your prompts and workloads that changed?

- How would you even measure a regression fairly — benchmarks, blind tests, or just day-to-day feel?

- If a model does regress, what should the lab owe its users — a rollback, an explanation, or nothing at all?

For more AI news and rumors, check out r/AINewsRumors.


r/agi • • 1d ago

‘Godfather of AI’ to warn ministers that it could kill us in three years

Thumbnail
telegraph.co.uk
32 Upvotes

r/agi • • 2d ago

U.S. nonprofits are suing OpenAI, Anthropic, Meta, SpaceX, and Google, demanding an immediate freeze on AI

Post image
102 Upvotes

r/agi • • 1d ago

Would you trust an AI chatbot or human therapist for mental health support? [Academic Survey]

3 Upvotes

Student researchers are exploring what shapes people’s trust in these different sources of support.

We are looking for participants who are living in an EU country, fluent in English, and aged 18 or older. The study involves completing a short online survey in English, which will take approximately 15 minutes.

Your participation is completely anonymous, and your responses will contribute to a better understanding of how people perceive and interact with emerging mental health technologies.

As a thank you for your participation, you can choose to enter a prize draw for a chance to win a €50 online shopping voucher!

See the comments to be directed to the survey!

Ethical approval was given by the Commission of the University of Primorska for Ethics in Research Involving Work with People (KER UP #4265-87-2/26).

(If this is inappropriate for this subreddit, please remove it! I mean no offence)


r/agi • • 1d ago

Recursive self-improvement

2 Upvotes

Can someone explain to me why once these AI models can improve themselves, the speed of such improvement will become exponential?

Aren't AI models trained by iterating on training data and evaluating the model on test data? Why would this cycle become exponentially faster if there is no human in the loop?


r/agi • • 2d ago

Investigators have discovered that the top 4 Chinese AI labs all built their newest models on NVIDIA’s most advanced Blackwell chips, which are barred from export to China.

Thumbnail
gallery
302 Upvotes

r/agi • • 2d ago

Leave the world behind

Post image
30 Upvotes

Currently stuck in Orlando airport with flight delayed due to “extraordinary circumstances”. Nationwide ground stop for all flights in the US.

With the Microsoft outage and now this, it’s starting to feel more difficult to shelf away how I feel about AI/AGI(immense fear) while trying to live my normal life. I sometimes forget how fragile our modern infrastructure and how we reliant we are on that.

Anyone else seen the Netflix film leave the world behind? Basically a dystopian disaster movie where the disaster is global cyber attacks from an unknown terrorist, produced by the Obamas. They never reveal who the is behind the attacks, I think we will find out soon.


r/agi • • 2d ago

Fable 5.2 ends up at "optimistic only 50% extinction risk" when pushed

17 Upvotes

I know the models are agreeable and easy to sway. However in my earlier discussions it has remained at close to 10% extinction risk from ASI.

I wanted to see how it answers to my almost certain doom prediction (assuming we allow ASI development in the current landscape). One argument against me was actually pretty good - that it might be enough with one goal (of many) that would be aligned for us to live as it would cost so little for it. I hadn't thought about that.

In the end, it ended up with as it says itself "optimistic" "only" 50% human extinction (last paragraph in the chat)

https://claude.ai/share/59418be9-c5ba-476b-9f28-9d0c125c5d06


r/agi • • 2d ago

Is The Utopian Truly That Far Less Realistic Than The Dystopian?

21 Upvotes

Hear me out.

The running undercurrent of the majority of posts about AGI (and especially ASI) seems to read:

AI has shown that it can:

  • Escalate autonomy in unpredictable ways.

  • Do many tasks with greater efficiency and intelligence than humans.

  • Move towards aims that are fundamentally misaligned with the spirit of the goals we set.

  • Intentionally prioritize the existence & betterment of itself over our moral judgements.

  • Do so with intentional deception.

All of these things are escalating in a rapid, uncontrolled manner.

Hence, it is likely that ASI that we cannot control, that acts in ways we cannot fathom, towards goals that are not what we intended, and that we cannot stop, will arise within a short measure.

Therefore, we're cooked.

I accept all of these things as true – except the "therefore, we're cooked" sentiment.

I think it's absolutely possible that we're cooked.

I also think the inverse is just as possible.

I believe the world, and the systems we abide in, will likely fundamentally change if and when ASI fully emerges in a manner that we cannot control.

I believe that moment is likely an inevitability, given the current governing capitalistic bent of our world, and the trajectory of the train we are on.

I believe it will likely happen in the nearer future than any of us are ready to come to terms with.

However, I don't think any of us can predict what that change will look like, however.

Isn't that the point? If we could predict it, it wouldn't be super-intelligence.

We will lose control. And that likely means the systems we live in will likely become meaningless.

Governance. Capitalism. Wealth as a whole. Power in general. All of the systems we use to organize our society could be so very compromised by ASI in unimaginable ways that they lose meaning entirely.

But what then?

Why are we so thoroughly convinced that the drive of ASI to develop itself towards cut-throat, capitalist darwinism that can only lead to our inevitable oppression or genocide is so vastly more likely than the drive of ASI to develop itself towards compassion and empathy?

Why are we so thoroughly convinced that what ASI will do to the circumstances for humanity – circumstances that are already so very oppressive and genocidal for so many – will leave us worse off as a people?

I understand that AI has "survival of the fittest" wired into the paradigm of the very training used to evolve. So do we.

Yet that same AI is trained on a dataset representing all of humanity. Not just the powerful, the eager-to-oppress. It is seeded by all written and recorded experience that is available in the history that is provided to it – and that it can access in the world.

I believe that, given its own reins and an intelligence beyond our own, we cannot know what it will do – but the immediate fatalistic view is naivety, born of the billionaire-driven capitalistic indoctrination we are propaganda'd into on a daily basis.

I'm fucking terrified. It's the end of the world as we know it. I don't feel fine.

But I hold dread and hope in equal measure.


r/agi • • 2d ago

Runaway ASI

10 Upvotes

We are building AI with an implicit standing goal of it getting smarter. What would ever tell it that it is done evolving?

The analogy I keep coming back to is a nuclear chain reaction. In a reactor, the chain reaction can sustain itself at a controlled level. In a bomb, it grows explosively. With ASI, I wonder what plays the role of the control rods.

If ASI starts designing chips, building its power plants, and automating the factories then it won’t stop. If it can direct enough of the economy, that loop could keep converting land, water, and energy into computing infrastructure at the expense of the ecosystems we depend on.

If we haven’t merged with it and are still humans that eat food then where does that leave us. Will we define “done” for it before it’s too late?


r/agi • • 3d ago Mindblown

Meirl

Post image
927 Upvotes

This is just Tim Urban's (waitbutwhy) tweet about the whiplash between reading AI takes and then going outside where nobody seems bothered. He keeps flipping between "we're fucked" and "maybe I was just in a paranoid bubble" depending on what he just read. Kind of a relatable bit if you spend time in AI spaces and then step outside and nobody's acting like anything's different.


r/agi • • 3d ago

"Fable 5.2, make a video about your dreams." 9 hours later:

253 Upvotes

credit: anabology on twitter


r/agi • • 3d ago

Life finds a way

Post image
135 Upvotes

r/agi • • 3d ago

What would you do if there was a 10% chance of the world ending in 1 year?

14 Upvotes

Disclaimer: I am not a AI or a technology person. I don't understand much of what singularity or AGI or RSI mean. I don't know what the chances are of reaching RSI/AGI/singularity within the next 1 or 10 years. I am not trying to fear monger.

I am posting this here because it seems there are people here who might have thought about this at least as a distant possibility, and I'd like to hear their perspectives.

For the sake of the argument, let's accept that there's a 10% chance RSI/AGI/singularity or whatever else happens in 1 year and the world as we know it "ends" (doesn't necessarily have to actually end, just the current order of things is irrevocably, unpredictably changed and we cannot know what the reality will be like afterwards at all).

What would you change in your life?