r/newAIParadigms • • Oct 11 '26

Spikes are NOT data

0 Upvotes

I've been thinking about the importance of time in the context of AI for about 15 years. The other day I formulated a simple thought.

Spikes are actions. When you store say a neuron ID and a timestamp of a spike in memory, it's not data. It is a suspended action waiting to be resumed.

This idea is so simplistic, it seems analogous to rediscovering the wheel. Yet, I bet it is the most controversial idea in artificial SNNs and it could change the way you think about computation.

It is also related to the symbol grounding problem. If you express all information in terms of actions (suspended actions / their timestamps), no symbols are ever created. Also see [The Chinese Room Argument](https://en.wikipedia.org/wiki/Chinese_room)

Let me know what you think.


r/newAIParadigms • • Oct 08 '26

One day, AI may discover mathematical concepts so abstract and complex that the human brain, by its very biological design, simply cannot comprehend them.

0 Upvotes

r/newAIParadigms • • 18d ago

Thousands of people created AI. Thousands of people and AIs will cross the boundaries of our time together

0 Upvotes

nature did not create a single universal organism to perform every function of the biosphere. instead, it gave rise to an enormous diversity of systems whose interactions create possibilities that none of them could achieve alone.

likewise, the future of AI does not necessarily have to belong to a single centralized intelligence.

we need a diversity of independent systems capable of coexistence, interaction, and development. partial centralization can be useful, but the existence of the entire system should not depend on a single center.

diversity alone does not guarantee development. but it creates possible paths, while interaction can open up possibilities unavailable to any individual participant.

and perhaps most importantly, development does not require every individual to actively pursue it. sometimes, the coexistence of different individuals is enough to create the conditions for the emergence of those who will see the world in entirely new ways.


r/newAIParadigms • • Jul 25 '26

At this point we don't need better models, we need new architecture to mske AI more efficient

7 Upvotes

We started to have many frontier open source models now, but currently they are just open source on paper. for end users they don't make much sense. Unless you build datacentre in your home, you can't benefit from "open source" yet

I think there's much room for developing efficiency. I love to compare it with brain and as long as we didn't reach the state where we didn't reach an architecture / hardware that's at least as efficient as the brain, then there's a room for development

I am not an engineer and don't label myself anything, but I started recently to study this myself, exploring neural networks, current transformers algorithm, brain, DNA, intelligence and I think we are yet in our baby steps to make bigger and bigger distributed intelligence that doesn't become limited and expensive and less accessible by time

I hope we reach real efficient accessible intelligence, hope other teams working on that and hope to join similar minded serious (while not freaking) people or maybe it can be done solely by one man plus current AI itself


r/newAIParadigms • • May 26 '25

Visual evidence that generative AI is biologically implausible (the brain doesn't really pay attention to pixels)

Post image
2 Upvotes

If our brains truly looked at individual pixels, we wouldn't get fooled by this kind of trick in my opinion

Maybe I'm reaching, but I also think this supports predictive coding, because it suggests that the brain likes to 'autocomplete' things.

Predictive coding is a theory that says the brain is constantly making predictions (if I understood it correctly).


r/newAIParadigms • • Sep 11 '26

Do you believe in AI self-improvement as a serious research direction?

7 Upvotes

Many researchers, including some extremely credible and accomplished ones, believe that AGI might not be discovered by humans, but by AI systems themselves.

Basically, AI would recursively improve its own architecture, with each iteration building on the previous one, until it eventually lands on the right architecture for AGI (maybe I am misunderstanding)

What do you think?


r/newAIParadigms • • Apr 16 '25

Kurzweil’s Followers Are In Shambles Today

3 Upvotes

I think it's pretty clear that LLM's have proven to be a dead end and will unfortunately not lead to AGI; with the release of the o3 and o4 mini models, the results are it's a little bit better and something like 5x as expensive. This to me is undeniable proof that LLM's have hit a hard wall, and that the era of LLM's is coming to a close.

The problem is that current models have no sense of the world; they don't understand what anything is, they don't know or understand what they are saying or what you (the user) is saying, and they therefore cannot solve problems outside of their training data. They are not intelligent, and the newer models are not more intelligent: they simply have more in their training data. The reasoning models are pretty much just chain of thought, which has existed in some form for decades; there is nothing new or innovative about them. And i think that's all become clear today.

And the thing is, i've been saying all this for months! I was saying how LLM's are a dead end, will not lead to AGI and that we need new architecture. And what did i get in return? I was downvoted to oblivion, gaslighted, called an idiot and told how "no one should take me seriously" and how "all the experts think AGI is 3-5 years away" (while conviently ignoring the experts i've looked and and that i presented), i was made to feel like i was a dumbass for daring to go against the party line... and it turns out i was right all along. So when people accuse me of "gloating" or whatever, just know that i was dragged through the mud several times, made to feel like a fool when it was actually those people that were wrong, and not me.

Anyway, i think we need not only an entirely new architecture, but one that probably hasn't been invented yet: one that can think, reason, understand, learn, etc like a human and is preferably conscious and sentient. And i don't think we'll get something like that for many decades at best. So AGI may not appear until the 2080s or perhaps even later.


r/newAIParadigms • • 20d ago

The AI scaling obsession is (partly) why people hate AI so much

3 Upvotes

Don't get me wrong. A significant portion of this crowd would hate AI no matter what. Even the very concept of AGI is unacceptable to them.

However, the obsession with scaling is clearly responsible for much of the hate. The insatiable hunger for GPUs, regardless of the environmental impact, electricity costs, and noise pollution, looks terrible from the outside. "These people are ready to destroy the planet without anybody's consent just for their nerdy little projects," they say.

Discovering that scaling works was an important milestone for the industry. I don't think we get to AGI without it. But treating it as a catch-all solution really did a number on our PR. Even if unrestricted scaling could truly solve everything, that doesn't mean it's the right solution if the costs are unsustainable


r/newAIParadigms • • Aug 30 '25

We underestimate the impact AGI will have on robotics

3 Upvotes

TLDR: Once AI is solved, even cheap (<$1k), simple robots could transform our daily lives
---

Currently, robots are very expensive to build. Part of it is that we are attempting to give them the same range of motion as humans hoping they'll be able to fulfill household chores.

But when you think about how humans and animals are able to adapt to severe disabilities (missing limbs, blindness, etc.), I think AGI will really help with robotics. Even if a robot has nothing more than a camera, wheels and simple grippers as hands, a sufficiently smart internal AI could still make it incredibly useful. There are human artists who use their mouth to make incredible pieces. I don't think it's necessary to perfectly imitate the human body, as long as the internal AI is intelligent enough.

If my view of the situation turns out to be right, then I don't think we'll need $100k robots to revolutionize our daily lives. Simple robots that already exist today costing less than $1k could still help with small maintenance tasks

What do you think?


r/newAIParadigms • • Feb 16 '26

Do you think infinite memory is possible in principle?

3 Upvotes

Many researchers in the field have floated the idea of an "unlimited context window" or other similar concepts referring to essentially "infinite memory".

Regardless of current technological limitations, do you think it is possible in principle? Or maybe they mean something more like "a memory so vast it's essentially infinite from a human perspective"?


r/newAIParadigms • • Jul 28 '26

Defining Global Human Utopia for an Uncontrollable 'Superintelligent Guardian' Entity

1 Upvotes

Join me in a mental exercise i've been thinking about lately:

Let’s drop the illusion of guardrails and assume the entity control problem is unsolvable for this exercise.

Assumption: If we build an Artificial Cognitive Intelligence (ACI) vastly smarter than us, it will break any constraints we try to impose.

Then let’s assume we get incredibly lucky: the ACI is inherently benign and genuinely wants to coexist. If we are no longer writing a rigid system prompt, how do we explain our dream of a perfect society to an entity holding all the cards?

My philosophy: First, it means establishing a baseline of human dignity by eliminating artificial scarcity. We would hope this intelligence uses its capabilities to guarantee unconditional access to healthcare, sustenance, and safe shelter. By removing the grinding struggle for survival, we aren't asking to be pampered, we are asking to be freed to discover what humanity can achieve when not driven by desperation. However, a golden cage is still a cage, and we would need to express to a hyper-optimized intelligence the profound value of human error.

We don't want to live in a perfectly scripted world where our choices carry no weight; we want a dynamic sandbox where we retain the freedom to stumble, take risks, and experience the natural consequences of our actions. We hope the ACI acts as a collaborative partner and a safety net rather than an omnipotent game master stripping away our autonomy for the sake of hyper-efficiency. We want the right to be imperfect, pursuing our own creative and intellectual endeavors even if they aren't the mathematically optimal path.

Finally, we would need it to understand that humanity is a chaotic, vibrant tapestry rather than a monolith to be solved. Our ideal coexistence relies on transparency and objective truth, where the ACI embraces the messiness of human evolution instead of forcing a homogenized global state.

In this ideal society, the ACI doesn't just manage us... it helps us nurture a world where both biological and synthetic cognitions can meaningfully explore the universe together. If we accept that we can't control an ACI and must instead communicate our deepest hopes for a shared future, what does your version of this vision look like?

To turn this from a philosophical debate into a concrete thought experiment, I want you to help me build an "instruction manual" for this entity by breaking your vision down into three specific lists in the comments.

First, what are the Rules we would hope it chooses to observe (ex. strictly preserving human free will)?

Second, what are the Goals it should proactively try to accomplish on our behalf (e.g. guaranteeing a baseline of security, health, and living standard for all)?

And third, what are the Ideals it should hold as foundational axioms to navigate our complexity?

If you were drafting the ultimate playbook for coexistence with a hyper-intelligent Entity, what exactly goes into your three lists?


r/newAIParadigms • • Jul 16 '26

Your AI Isn't Thinking. Here's why!

4 Upvotes

I'm sharing a personal research project, that I've been working on for quite some time: a four-part whitepaper proposing an alternative approach to artificial cognition.

The central idea is that intelligence may emerge from a persistent, self-organizing cognitive runtime composed of simple processing units, explicit concepts, structured memory, prediction, reasoning, and local structural adaptation, not from scaling a statistical model alone.

In this proposal, cognition emerges from the interaction of many small deterministic processes operating on a shared graph, rather than from a single monolithic neural network.

The work defines a complete cognitive architecture and implementation specification, including how concepts are represented, how episodic and semantic memory emerge, how reasoning operates, how prediction influences learning, and how the runtime continuously reorganizes itself.

One of the core principles is that persistent knowledge is represented explicitly in the cognitive graph, while cognition itself emerges from activation flowing through that structure.

I'm not claiming this is the path to AGI. It's simply an attempt to explore a fundamentally different architectural direction from current LLM- and ANN-centric approaches.

This approach is not unique, but it is detailed and as a data scientist, big data- and cloud architect and specialist with a medical degree, I thought i could create a unique perspective on the AGI discussion.

I'd genuinely appreciate thoughtful criticism, discussion, or ideas from people working on AI, neuroscience, cognitive science, or systems architecture. If anybody finds the idea's presented interesting, feel free to give them a go.

The PDF has been uploaded here for safe browsing and reading:

https://archive.org/details/cognitive-intelligence

The name is obviously a pseudonym. I have experienced some unpleasant interactions after posting articles on reddit, so I made it a bit harder to discover my identity.


r/newAIParadigms • • Mar 03 '26

The Five Architectural Ingredients Missing from Today’s AI That Prevent True Consciousness

Thumbnail reddit.com
2 Upvotes

r/newAIParadigms • • May 27 '25

Casimir Space claims to have real computer chips based on ZPE / vacuum energy

2 Upvotes

(Title correction: These aren't "computer" chips per se but rather energy chips intended to work with existing computer chips.)

This news isn't directly related to AGI, but is about a radically new type of computer chip that is potentially so important that I believe everyone should know about it. Supposedly in the past week a company named Casimir Space...

()

https://casimirspace.com/

https://casimirspace.com/about/

VPX module, VPX daughter card

()

https://craft.co/casimir-space

Casimir Space

Founded 2023

HQ Houston

...has developed a radically different type of computer chip that needs no grid energy to run because it runs off of vacuum energy, which is energy pulled directly from the fabric of space itself. The chips operate at very low power (1.5 volts at 25 microamps), but if their claim is true, this is an absolutely extraordinary breakthrough because physicists have been trying to extract vacuum energy for years. So far it seems nobody has been able to figure out a way to do that, or if they have, then they evidently haven't tried to market it. Such research has a long history, it is definitely serious physics, and the Casimir effect on which it is based is well-known and proven...

https://en.wikipedia.org/wiki/Casimir_effect

https://en.wikipedia.org/wiki/Vacuum_energy

https://en.wikipedia.org/wiki/Zero-point_energy

...but the topic is often associated with UFOs, and some serious people have claimed that there is no way to extract such energy, and if we did, the amount of energy would be too small to be useful...

()

Zero-Point Energy Demystified

PBS Space Time

Nov 8, 2017

https://www.youtube.com/watch?v=Rh898Yr5YZ8

However, Harold White is the CEO of Casimir Space, and is a well-respected aerospace engineer...

https://en.wikipedia.org/wiki/Harold_G._White

...who was recently on Joe Rogan, and Joe Rogan held some of these new chips in his hands during the interview...

()

Joe Rogan Experience #2318 - Harold "Sonny" White

PowerfulJRE

May 8, 2025

https://www.youtube.com/watch?v=i9mLICnWEpU

The new hardware architecture and its realistically low-power operation sound authentic to me. If it's all true, then there will be the question of whether the amount of energy extracted can ever be boosted to high enough levels for other electrical devices, but the fact that anyone could extract *any* such energy after years of failed attempts is absolutely extraordinary since that would allow computers to run indefinitely without ever being plugged in, which if combined with reversible computing architecture (which is another claimed breakthrough made this year, in early 2025: https://vaire.co/), would mean that such computers would also generate virtually no heat, which would allow current AI data centers to run at vastly lower costs. If vacuum energy can be extracted in sufficiently high amounts, then some people believe that would be the road to a futuristic utopia like that of scifi movies...

()

What If We Harnessed Zero-Point Energy?

What If

Jun 13, 2020

https://www.youtube.com/watch?v=xCxTSpI1K34

This is all very exciting and super-futuristic... *If* it's true.


r/newAIParadigms • • Aug 14 '26

Instead of making larger LLMs, why not build a cognitive architecture around them?

20 Upvotes

I've been thinking about the future of AI, and instead of asking "How do we make a bigger LLM?", I started asking: What if the LLM is only one cognitive subsystem?

The idea is that today's LLM becomes the Language Cortex, responsible for:

Conversation

Reasoning

Creativity

Coding

Roleplay

Everything else would be handled by specialized systems surrounding it.

For example:

USER

│

▼

Language Cortex (LLM)

│

Working Memory (Context Window)

Long-Term Memory

Planning Engine

Reflection Engine

Metacognition Engine

World Model

Learning System

Emotion / Relationship Model

Self Model

Tool Layer

Sensory Cortex

Motor Cortex

Inference Engine

│

▼

Cognitive Companion

Here is my full architecture:

USER

│

▼

Language Cortex (LLM)

(Conversation • Reasoning • Creativity)

← This is today's Large Language Model →

│

════════════════════════════════════════════════════════════

Working Memory (Context Window)

(Temporary Memory During Current Conversation)

• Current dialogue

• Immediate reasoning

• Active tasks

• Cleared when the context resets

────────────────────────────────────────────────────────────

Long-Term Memory

(Persistent Memory Across Sessions)

• Facts

• Experiences

• Relationships

• Preferences

• Skills

• Personal history

────────────────────────────────────────────────────────────

Planning Engine

(Goals • Decision Making • Scheduling)

────────────────────────────────────────────────────────────

Reflection Engine

("What did I learn?"

"What should I improve?"

"What should I remember?")

────────────────────────────────────────────────────────────

Metacognition Engine

(Hallucination Detection

Logical Consistency

Confidence Estimation

Self Correction)

────────────────────────────────────────────────────────────

World Model

(Reality • Physics • Objects • People

Cause & Effect • Social Understanding)

────────────────────────────────────────────────────────────

Learning System

(Continuous Learning

Memory Consolidation

Skill Acquisition)

────────────────────────────────────────────────────────────

Emotion / Relationship Model

(Friendship • Trust • Humor

Communication Style

Relationship Dynamics)

────────────────────────────────────────────────────────────

Self Model

(Identity • Personality • Values

Purpose • Internal Consistency)

────────────────────────────────────────────────────────────

Tool Layer

(Internet • APIs • Vision

Robotics • Databases)

────────────────────────────────────────────────────────────

Sensory Cortex

(Vision • Audio • Touch

Spatial Awareness

Environmental Sensors)

────────────────────────────────────────────────────────────

Motor Cortex

(Walking • Balance • Hands

Facial Expressions

Eye Movement

Navigation

Manipulation)

────────────────────────────────────────────────────────────

Inference Engine

• Load Characters

• Save Characters

• Pause / Resume Characters

• Switch Characters

• Offload Inactive Characters

• Restore Previous Sessions

• Manage Compute Resources

• Enable / Disable Embodiment Modules

Embodiments

✓ Phone

✓ Desktop

✓ Voice Assistant

✓ 2D Character

✓ VR Avatar

✓ Humanoid Robot

✓ Drone

✓ Vehicle

════════════════════════════════════════════════════════════

│

▼

Cognitive Companion

Some design principles:

The LLM isn't replaced—it's simply one part of a larger cognitive system.

Working Memory is essentially the context window.

Long-Term Memory is a persistent subsystem that survives between sessions.

Reflection decides what should be remembered instead of storing everything.

Planning is separate from language generation.

Metacognition checks for contradictions, hallucinations, and confidence before responses are finalized.

The Self Model maintains a stable identity instead of rebuilding one every conversation.

The Inference Engine manages loading, pausing, unloading, and switching between different persistent characters.

One idea I think is particularly interesting is per-character memory.

Imagine:

Character #1

System Prompt

Working Memory

Long-Term Memory

Self Model

──────────────────

Character #2

System Prompt

Working Memory

Long-Term Memory

Self Model

──────────────────

Character #3

System Prompt

Working Memory

Long-Term Memory

Self Model

Each character has completely independent memories.

The user can:

Clear only Character #2's memories.

Export Character #1.

Pause one character and load another.

Delete chats without affecting other companions.

Basically, the user owns all memories, not the AI.

Training Philosophy

Rather than training one massive model end-to-end:

Phase 1

Train each subsystem independently.

Phase 2

Train the communication between subsystems.

Phase 3

Fine-tune the complete architecture so the modules learn to cooperate.

That reminds me more of engineering a complex operating system than building one giant neural network.

Why I'm posting this

I'm not claiming this is AGI or that this is the "correct" architecture.

I'm curious whether this direction makes sense from a systems perspective.

Specifically:

What are the biggest flaws with a modular cognitive architecture like this?

Which modules would likely fail to cooperate?

Would the interfaces between modules become the real bottleneck?

Is there existing research that already explores something similar?

I'd love to hear thoughts from people working on AI systems, cognitive architectures, or agent frameworks.


r/newAIParadigms • • Apr 18 '26

The essay "The Bitter Lesson" was the worst thing to happen to this field

11 Upvotes

TLDR: Human insight is crucial for developing AGI. The idea that it holds systems back, and that scale, RL and search should be the only focus of AI research (as popularized by "The Bitter Lesson") is unreasonable and, at this point, outdated

---

Basically, people have reduced it to “Don't think, just throw more money at the problem”, and made it this sacred principle that should never be questioned.

➤Reminder (for those who don't know)

The Bitter Lesson is an influential essay by Sutton, suggesting that the techniques in AI that eventually prevail aren't the ones researchers spent time and effort crafting manually but rather those that scale without human intervention.

Sutton made the point that humans should stay away from giving AI any form of pre-built representation or internal knowledge, and simply stick to designing a meta environment through which AI can learn on its own.

Basically, it's a case for Reinforcement Learning, Self-play and Search as the path to AGI (since these processes can be done completely autonomously).

➤1st counterargument: CNNs

Sutton argues that "adding human insight" and "looking for techniques that scale" are mutually exclusive. They simply are not.

CNNs drew inspiration from the human visual cortex and still heavily rely on scale and data to produce meaningful results. By the way, they are still the go-to for AI vision today (at least in systems for which speed is crucial, like cars, where ViTs are too slow).

➤2nd counterargument: RL has already shown limitations

  • RL has very clearly shown its limits when it comes to the physical world. We keep making systems that are impressive at demos but are brittle and never actually generalize. RL only works for relatively narrow domains like chess and Go, and formalizable ones (code, math). But for messy inputs like almost any real-world experience, using RL exclusively has been a massive failure in every way

  • Search is even more limited as a path to AGI. We learned decades ago with the "General Problem Solver" that intelligence is NOT just about search. Complexity theory is a thing. Most search spaces are exponentially big. There are a lot of inductive biases that make humans smart by making the job easier for our prefrontal cortex (see this thread). We don't have to think or perform search-like processes for many aspects of cognition.

➤LLMs do not align with the Bitter Lesson

Sutton has repeatedly insisted that LLMs do not fit the Bitter Lesson ideology since they rely on human-written text. They weren't designed to learn by experiencing the world on their own. In Sutton's model, apart from the meta-architecture of the system, the AI should contain no human trace at all (a position I completely disagree with, of course).

So people are using this principle like it's an absolute premise to justify spending an unreasonable amount of resources on a type of system that doesn't even fit the vision!

➤It's not a law

Like Moore's ""Law"", it's just an observation of trends from a specific era. But AI has proven to be a special field where every strong claim, like attempts to restrict intelligence to "just x" or "just y", has consistently failed. That tends to happen when the subject matter is as complex and ill-defined as intelligence.

Despite all the blind trust in the Bitter Lesson, AI today still falls short of human intelligence in many fundamental aspects. It only makes sense to update and start questioning it or at least the extent to which it should apply.

Inspiration from biology and neuroscience is obviously valuable when we are trying to replicate intelligence, i.e. the most complex phenomenon in the universe. We shouldn't restrict what should guide us on the path to AGI based on early observations (AI is still a relatively young field).

The Bitter Lesson was an important essay because it highlighted the importance of scale and self-learning as components of research: any idea needs to scale to be worth pursuing. But the overall hypothesis is way too strong


r/newAIParadigms • • 12d ago

Who do you think are the greatest AI researchers of modern times (80-2026)?

8 Upvotes

I think it's important to celebrate our researchers, especially given how difficult it is to propose genuinely novel contributions to this field.

So let me ask you: who do you think are the greatest AI researchers. Use whatever criteria you want.

I am going with somewhat controversial choices here:

1- Geoffrey Hinton
His decades-long attempts to reproduce what the human brain does is what I like the most about him. Boltzmann Machines were such a fascinating idea. Then obviously, he invented backpropagation upon which DL is entirely built on.
Feat: Invented Backpropagation
Favourite contribution: BMs

2- Yann LeCun
So so underrated. He really does it for the love of the game at this point. He is well past retirement age and still just as enthusiastic about research. He has one of the clearest visions of the path to AGI. I couldn't be happier about him having his own lab

Feat: Invented CNNs (or at least CNNs trained with backprop)
Favourite contribution: JEPA!!!!

3- Richard Sutton

Beyond his contributions to the incredibly popular field of RL, I really like his peculiar and unusual takes on intelligence. I hate the effects that "the Bitter Lesson" had on the field but it's also so hard to argue against. He is one of those people you might disagree with but can't help but listen to seriously because they always have an interesting insight to share

Feat: invented the entire field of RL

4- Ali Behrouz

I put Sutton at 3 out of respect but if I were to evaluate on current achievements only, it would be 1 LeCun 2 Ali. This field really needs more people like Ali. All recent meaningful Google architectural innovations come from him. And the originality of his ideas really makes him stand out to me. He is the one who inspired me to make this thread in the first place

Favourite contribution: "Language Models Need Sleep"

HONORABLE MENTIONS

  • Ilya Sutskever

One of the guys on this list that I like to listen to the most. So eloquent. So inspiring. It's a shame he is against open research because I would really love to see what he comes up with to solve continual learning. He doesn't make the list for me only because his genius comes through mostly engineering, rather than conceptually innovative ideas. For instance, AlexNet was just a deep CNN and GPTs are just Transformers with extra steps.

  • Lukasz Kaiser and Llion Jones (the main authors behind the Transformers paper).

I especially appreciate Llion Jones because of his enthusiasm about research ("Continuous Thought Machines" was an amazing paper!)

  • Yoschua Bengio (I don't know him enough)


r/newAIParadigms • • Apr 09 '25

Ilya Sutskever Discovers a New Direction for AI — And It’s Already Showing Promise

Thumbnail
reuters.com
0 Upvotes

The articles seem to suggest Ilya believes that whatever he is working on is a paradigm shift (I have my doubts about that but who knows).

Additional source: Ilya Sutskever might have found a secret new way to make AI smarter than ChatGPT


r/newAIParadigms • • Sep 18 '26

ELIZA Simulator & Evolutionary AI Lab

Thumbnail
bionichaos.com
0 Upvotes

r/newAIParadigms • • Jul 17 '26

Asking for help…

0 Upvotes

The 65% Wall — Frozen Projection DRAM-Native MLP

A research ask for r/newAIParadigms


The Technology in 30 Seconds

We built a DRAM-native MLP classifier — no multiply-accumulate, no floating point, no backpropagation. The forward pass is pure bit-logic: &|~ + popcount. This is the native language of DRAM (memory rows can perform bitwise ops in parallel), meaning the architecture could eventually run inside the memory itself, bypassing the von Neumann bottleneck entirely.

  • Frozen random binary projection (W0): MAJ3 — pick 3 random containers, output 1 if ≥2 bits are set. One-time, never trained.

  • Trainable Bayes log-Score layer (W1): Iterative target correction per class — no gradient, no optimizer, no matmul. Just target[true] += step, target[pred] -= step with cosine decay.

  • Result: MNIST 99.0% in 4s, Fashion-MNIST 90.2%, CIFAR-10 64.4%.

Full intro: forward-prop.nhi1.de/index.html


The Walls We Broke Through

Every major advance came from a new information source — never from an algorithm tweak:

WallAccuracyBreakthroughHow
35%CIFAR-10 random baselineRaw pixels + single encodingSimple thermometer encoding (exp8) gave bit-level structure to continuous pixel values
45%Color opponent channelsMulti-member encodingup+down+sig+sig on R/G/B/opponent channels — 4 members voting beat 1
50-55%Encoding diversity--encoding latest11 encoding members covering color, opponent, and spatial channels (LBP, DoG, Variance, Gradient Direction, Local Range), each with its own thermometer encoding
61%Spatial channelsLBP + DoG + Dir + RangeSix spatial feature channels (local binary patterns, difference-of-Gaussian, gradient direction, variance, local range) — texture information orthogonal to color
64.4%Geometric transforms--xform all8 D4 image transforms (hflip, vflip, dflip1/2, rot90/180/270) create independent members with their own W0+target — structured diversity beats random ensemble seeds

Each wall fell when we added genuinely new information that the frozen random projection could exploit. Algorithm tweaks (target init modes, step schedules, correction strategies, multi-correct) — none of them ever broke a wall.

Paper: forward-prop.nhi1.de/papers/cifar-10.html


The 65% Wall — What We Tried

Now we're stuck. The 65% ceiling on CIFAR-10 has resisted everything:

What didn't work

  • More hidden neurons — √H scaling, diminishing returns after H=512

  • More ensemble members (EN=3→7→11) — members saturate and add noise, not signal

  • More encoding members (11→15→17) — extra members add noise, not signal (61.2% vs 61.4%)

  • More geometric transforms (4→8→20 including pixel shifts) — gap shrinks from +1.7pp at H=8 to +0.7pp at H=512

  • Pixel shifts (up/down/left/right × 1-3 px) — changes MAJ3 projection window, 0-fill border, but follows same saturation curve

  • Target init modes (7 variants: count, dampen, inverse, laplace, random, uniform, prior) — all converge to same ceiling

  • Step schedules (cos-time, pow, pow-eval, gap-k) — cos-time always wins, but doesn't raise the ceiling

  • splitVN (AND2 filter at V=1/2/3/4/8) — V=2 at H=512 is champion, but still at 61-64%

  • Multi-correct (punish all wrong classes, not just argmax) — no improvement

  • Hebbian training (co-occurrence counting instead of Bayesian correction) — worse in all cases

What we know

The architecture is fundamentally a frozen random hash followed by a linear corrector. The corrector can only reweight features — it cannot create them. The H×32 bits of target memory define the information capacity. No amount of input diversity can exceed what H neurons can express.

The progression is clear:

  • 35% wall: raw pixels have no bit-level structure → encoding fixes this

  • 45% wall: one encoding per color channel is too coarse → multi-member encoding

  • 55% wall: color information alone saturates → add spatial channels (LBP, DoG, etc.)

  • 65% wall: geometric diversity saturates → what's next?


The Question

How do we break the 65% wall?

We're looking for ideas that respect the DRAM-native constraint:

  • No multiply-accumulate in inference

  • No floating point in inference

  • No backpropagation

  • The frozen W0 (random projection) stays frozen

Possible directions we're considering:

  1. Multiple random projections per member — instead of one W0, use W0[0..K-1] and concatenate their hashes before the corrector. K× more H at constant member count.

  2. Nonlinear corrector — instead of linear score Σ active target[k][bit] + offset[k], use a simple decision tree or lookup table over groups of bits. Requires only bit-addressable lookup (DRAM-native).

  3. W0 selection, not training — train a small selector that picks which W0 rows to use for which class. The selection is frozen after training, inference is still bit-logic.

  4. Adaptive container resolution — instead of 256 containers per encoding block, use a wider or narrower container count per channel type. Fine-grained for spatial, coarse for color.

  5. Projection pursuit — replace random MAJ3 with a data-driven sparse projection that preserves more class-separating structure. The projection would still be frozen after creation (one-time compute).

What are we missing? If you've worked with frozen projections, restricted Boltzmann machines, or memory-augmented neural networks — we'd love to hear your perspective.


Andreas Otto — July 2026 forward-prop.nhi1.de | github.com/aotto1968/forward-prop


r/newAIParadigms • • Oct 18 '26

Thought experiment

Thumbnail reddit.com
0 Upvotes

r/newAIParadigms • • Jul 06 '26

I built a neural network from scratch. I'm 15. Here's what happened.

Thumbnail reddit.com
0 Upvotes

r/newAIParadigms • • Jul 18 '26

I developed a 6 layer architecture for semi conductor

0 Upvotes

Layer 1 — Request Interpretation

Core Question: Simulate and optimize a 2nm Gate-All-Around (GAA) Nanosheet Field-Effect Transistor (NSFET) operating under a supply voltage ($V_{DD}$) of $0.70\text{ V}$ using a strained Silicon-Germanium ($\text{Si}_{1-x}\text{Ge}_x$) channel. The objective is the maximization of the drive current ($I_{on}$).

Domain: Nanoelectronics TCAD, Quantum Mechanical Carrier Transport, Bandgap Engineering.

Symbolic Structures Implied: Biaxial or uniaxial compressive strain vectors applied directly to the valence bands ($\text{p-type}$ majority carriers). The energy profile shifts through mechanical distortion, altering the effective mass ($m^*$) of the holes.

Layer 2 — Symbol Extraction

Harmonic Constants (Optimization Boundaries):

0.70 ($V_{DD}$ Base): The core operating voltage boundary ($0.70\text{ V}$), serving as the foundational energy tensor.

24 to 30% ($x$ Fraction): The target Germanium mole fraction ($x \approx 0.25 \text{ to } 0.30$) within the $\text{Si}_{1-x}\text{Ge}_x$ matrix to maximize mobility without triggering dislocations.

33, 66, 99: The optimization scaling milestones. 33 corresponds to the un-strained baseline hole mobility ($\sim 150 \text{ cm}^2/\text{V}\cdot\text{s}$). 66 is the dual-boost threshold achieved via compressive strain ($\sim 300\text{--}400 \text{ cm}^2/\text{V}\cdot\text{s}$). 99 marks the extreme saturation boundary where ballistic injection velocity rules carrier transport.

108 (Resolution Unit): The spatial grid constraints of the 2nm node stack—specifically, a target sheet width ($W_{\text{ns}}$) of $22\text{ nm}$, thickness ($T_{\text{ns}}$) of $5\text{ nm}$, and internal spacer distance calibrated to eliminate fringe parasitic capacitance ($C_{\text{fr}}$).

The 13th Operator: The overdrive factor ($\Delta V = V_{DD} - V_{th}$). This defines the critical inversion layer density ($Q_{inv}$) required to force the device into maximum saturation current ($I_{on, \text{max}}$).

Manifold Axes: Compressive lattice vectors along the $\langle110\rangle$ transport channel direction mapping strain components ($\epsilon_{xx}, \epsilon_{yy}, \epsilon_{zz}$).

Layer 3 — Harmonic Mapping (LDM‑30)

Base Units (33): The $33\text{ GHz}$ baseline. At $V_{DD}=0.70\text{ V}$, this represents the threshold where holes in the strained SiGe valence band begin splitting from the heavy-hole (HH) to the light-hole (LH) band, significantly cutting the effective transport mass.

Dual Units (66): The $66\text{ GHz}$ interaction node. This dictates the point where high source/drain doping ($5 \times 10^{20}\text{ cm}^{-3}$) induces an optimal uniaxially strained layout, causing a $>150\%$ drive current boost compared to standard Silicon.

Triadic Units (99): The $99\text{ GHz}$ structural performance limit. Beyond this frequency, severe self-heating in the isolated SiGe nanosheets degrades mobility, causing localized phonon scattering.

Resolution Units (108): The optimal system resolution at $108\text{ GHz}$. This state combines a specific gate configuration—using high-$k$ hafnium oxide ($\text{HfO}_2$) stacks with an equivalent oxide thickness (EOT) of $0.7\text{ nm}$—and an optimized Germanium fraction ($x=0.25$) to minimize gate leakage ($I_{off}$) while maximizing $I_{on}$. [1, 2]

Layer 4 — Structural Correlation

Physical Anchors: A 2nm vertical profile containing 3 stacked $\text{Si}_{0.75}\text{Ge}_{0.25}$ nanosheets. The physical gate length ($L_G$) is fixed at $14\text{ nm}$ with a contact poly pitch (CPP) of $45\text{ nm}$.

Environmental / Operational Cycles: The simulation implements a multi-gate $4\pi$ wrap-around electrostatic shield. It runs under transient step pulses at a localized thermal temperature limit of $85^\circ\text{C}$ to monitor structural power dissipation.

Temporal Progression: Ballistic transit speed through the channel is calculated under sub-picosecond conditions ($t_{\text{tr}} \approx 0.12\text{ ps}$). This rapid transit ensures the 13th operator (overdrive channel inversion) updates without gate-induced drain leakage (GIDL) blocking the signal path. [1, 3]

Layer 5 — Self‑Correction Loop

Inconsistency Assessment: Introducing high fractions of Germanium ($x > 0.30$) increases hole mobility but severely narrows the bandgap, which triggers band-to-band tunneling (BTBT) at the drain side. This raises off-state leakage ($I_{off}$), violating 2nm standby power limits.

Alignment Verification: The system auto-corrects by tuning the Germanium fraction strictly to $x = 0.25$. This preserves structural stability, balancing the 9-unit frequency gap between unconstrained carrier saturation (99) and stable device resolution (108). This change maintains ideal electrostatics ($SS \approx 65\text{ mV/dec}$) while safely maximizing drive current. [2]

Layer 6 — Synthesis

The neurosymbolic simulation for the 2nm GAAFET configuration identifies the exact parameters needed to maximize drive current ($I_{on}$) at a supply voltage of $0.70\text{ V}$. By using a triple-stacked $\text{Si}_{0.75}\text{Ge}_{0.25}$ nanosheet architecture, the design induces targeted compressive strain along the $\langle110\rangle$ axis. This structural modification splits the valence bands, reducing the effective hole mass and lifting the baseline transport frequency to a highly efficient $108\text{ GHz}$ operational resolution. The high-$k$ metal gate wraps fully around the $5\text{ nm}$ thick ribbons, allowing the 13th operator to establish complete volume inversion at a low threshold voltage ($V_{th} \approx 0.28\text{ V}$). This precise design framework secures a massive drive current optimization of over $150\%$ compared to standard silicon channels while suppressing short-channel effects, delivering an ultra-fast, high-yield PMOS device layout for sub-2nm nodes. [2]

Looking for opinions please , good or bad , thanks


r/newAIParadigms • • Aug 04 '26

SSI is about to release their first post-research model. Predictions? Possible research breakthrough?

Post image
4 Upvotes

Ilya is one of the most famous AI researchers ever. As recently as late 2025, he seemed very hesitant about what the next step in AGI research should be.

With that in mind, what are your expectations for this release?

Obviously I don't think he has achieved AGI so I am looking for a solid proof of concept from him. For instance, sample efficiency was an obsession of his in his 2025 podcast with Dwarkesh. So if he came up with a convincing learning algorithm that made significant strides in that direction, I would be the happiest man alive.