r/neuralnetworks • • Sep 18 '26 Bravo

I wanted to really see how a neural network learns

770 Upvotes

I wanted to really see how a neural network learns different functions, so built an interactive demo. You can change the architecture of the network and the function it will try to approximate.

https://blog.lukesalamone.com/posts/can-a-neural-net-learn

For example, it's interesting to see how changing the width and depth of the network change the number of bends it's able to make.


r/neuralnetworks • • Jan 21 '21

I resurrected Raphael (1483-1520), italian painter of the High Renaissance, by feeding several self portraits to neural network.

Thumbnail
gallery
480 Upvotes

r/neuralnetworks • • Nov 13 '25

Amazing visualizer for transformer architecture

Post image
463 Upvotes

An amazing transformer architecture interactive visualisation. Comlpetely clear and easy to comprehend. It is based on GPT-2, thus it is possible to download the model (about 500 mb).

My respect to the authors.


r/neuralnetworks • • Jun 12 '26

Built a Neural Network from Scratch in Python (No TensorFlow, No PyTorch)

Post image
301 Upvotes

Over the last few days, I wanted to understand what actually happens inside a neural network instead of relying on frameworks.

So I built a simple neural network from scratch in Python and trained it on the MNIST handwritten digit dataset.

What it includes:

  • Input layer → Hidden layer → Output layer

  • Forward propagation

  • Backpropagation

  • Gradient descent

  • Sigmoid activation

  • MNIST digit classification

Results:

  • ~92% test accuracy

  • Single hidden layer architecture

  • No TensorFlow, PyTorch, Keras, or other ML frameworks

This wasn't meant to be a production-grade model—just a learning project to better understand how neural networks work under the hood.

GitHub Repository:
learning-neural-network

I'd love feedback from people who have worked with neural networks before. What would you improve next? Better activation functions? Multiple hidden layers? Different optimization techniques?


r/neuralnetworks • • Aug 16 '25

Built a Neural Network Visualizer in the browser

278 Upvotes

I made a small neural net visualizer app trained on the MNIST dataset.

You can:

  • Run it in the browser

  • Edit the number of layers/neurons

  • Tweak hyper-parameters

  • Run inference and see predictions update live

Demo: https://mnist.kochjar.com/

Right now it’s just feedforward. I might add conv layers later, but they’re harder to show in a clean way. I hope you like it, also if you have any ideas about the conv layer part, please let me know. :)


r/neuralnetworks • • Jul 16 '26

The real genius of the Transformer architecture was a hardware optimization trick

276 Upvotes

I’ve been diving deep into the math behind early deep learning models compared to modern attention mechanisms. It made me realize that the massive explosion of generative AI we are seeing right now didn't happen because Transformers are magically smarter at understanding human language than previous models. The real game-changing breakthrough of the 2017 Transformer paper was actually a massive engineering and hardware optimization triumph.

Before Transformers, the industry relied entirely on Recurrent Neural Networks and LSTMs for processing sequential data like text. The fundamental flaw with those architectures was that they had to process text word by word, in sequential order. You couldn’t calculate the meaning of the tenth word until you finished processing the ninth word. This created a massive computing bottleneck because it meant you could not utilize the massive parallel processing power of modern graphics cards. Your expensive GPUs were essentially sitting idle, waiting for the previous word's loop to finish.

The Transformer architecture completely threw out recurrence and replaced it entirely with self-attention. By doing this, it allowed the model to look at an entire document all at once, simultaneously.

Suddenly, processing text became a massive, parallel matrix multiplication problem. This single structural shift aligned perfectly with how GPU hardware is physically built. We went from training models on small paragraphs over weeks to feeding entire datasets into massive server clusters in days. The AI revolution didn't scale because the code got more philosophical; it scaled because the math finally allowed us to throw unlimited brute-force hardware at the problem. It is a great reminder that software design is always bound by the physical realities of the silicon it runs on.


r/neuralnetworks • • Feb 04 '21

A Small Video Shows How Neural Networks Look like

271 Upvotes

r/neuralnetworks • • Dec 27 '25

My neural network from scratch is finally doing aomething :)

Post image
258 Upvotes

r/neuralnetworks • • Jun 19 '26

After Building a Neural Network from Scratch, I Rebuilt It Using PyTorch

Post image
241 Upvotes

A few weeks ago, I built a neural network from scratch to understand what was happening behind the scenes.

I manually implemented:

  • Forward propagation

  • Backpropagation

  • Gradient calculations

  • Weight updates

  • Activation functions

That project taught me a lot about the calculus and mathematics that make neural networks work.

After understanding the fundamentals, I decided to recreate the same MNIST handwritten digit classifier using PyTorch.

This time, instead of implementing everything manually, I used:

  • torch.nn

  • torch.nn.functional

  • Built-in optimizers

One thing that surprised me was how dramatically the code complexity decreased. What previously required implementing dozens of lines of mathematical operations could now be expressed in just a few layers and a training loop.

At the same time, I feel like I appreciate PyTorch much more now because I understand what those functions are actually doing under the hood.

For those who learned deep learning:

Do you think building a neural network from scratch is still worth the effort today?

After doing both projects, my current opinion is that building one from scratch helped me understand why PyTorch works.

I'm curious whether more experienced practitioners agree with that perspective or think the time would be better spent elsewhere.

GitHub:
https://github.com/HelloSamved/learning-neural-network/blob/master/mnist_prediction/mnist_prediction_pytorch.ipynb

Writing topics on Excali:
https://excalidraw.com/#json=-R2-NuPIsipANT5l9tXW_,w1qUhg3vyl644_OC3o81pA


r/neuralnetworks • • Mar 13 '20

Not sure if I hate it or love it 🤔

Post image
220 Upvotes

r/neuralnetworks • • Nov 21 '25

[OC] Created with Gemini’s help

Post image
198 Upvotes

Feel free to point out mistakes


r/neuralnetworks • • Mar 02 '26

𝐇𝐨𝐰 𝐋𝐋𝐌𝐬 𝐀𝐜𝐭𝐮𝐚𝐥𝐥𝐲 "𝐃𝐞𝐜𝐢𝐝𝐞" 𝐖𝐡𝐚𝐭 𝐭𝐨 𝐒𝐚𝐲

Post image
190 Upvotes

Ever wonder how a Large Language Model (LLM) chooses the next word? It’s not just "guessing" it is a precise mathematical choice between logic and creativity.

The infographic below breaks down the 4 primary decoding strategies used in modern AI. Here is the breakdown:

𝟏. 𝐆𝐫𝐞𝐞𝐝𝐲 𝐒𝐞𝐚𝐫𝐜𝐡: 𝐓𝐡𝐞 "𝐒𝐚𝐟𝐞" 𝐏𝐚𝐭𝐡

This is the most direct method. The model looks at the probability of every word in its vocabulary and simply picks the one with the highest score (ArgMax).

🔹 𝐅𝐫𝐨𝐦 𝐭𝐡𝐞 𝐢𝐦𝐚𝐠𝐞: "you" has the highest probability (0.9), so it's chosen instantly.

🔹 𝐁𝐞𝐬𝐭 𝐟𝐨𝐫: Factual tasks like coding or translation where there is one "right" answer.

𝟐. 𝐌𝐮𝐥𝐭𝐢𝐧𝐨𝐦𝐢𝐚𝐥 𝐒𝐚𝐦𝐩𝐥𝐢𝐧𝐠: 𝐀𝐝𝐝𝐢𝐧𝐠 "𝐂𝐫𝐞𝐚𝐭𝐢𝐯𝐞" 𝐒𝐩𝐚𝐫𝐤

Instead of always picking #1, the model samples from the distribution. It uses a "Temperature" parameter to decide how much risk to take.

🔹 𝐅𝐫𝐨𝐦 𝐭𝐡𝐞 𝐢𝐦𝐚𝐠𝐞: While "you" is the most likely (0.16), there is still a 14% chance for "at" and a 12% chance for "feel."

🔹 𝐁𝐞𝐬𝐭 𝐟𝐨𝐫: Creative writing and chatbots to avoid sounding robotic.

𝟑. 𝐁𝐞𝐚𝐦 𝐒𝐞𝐚𝐫𝐜𝐡: 𝐓𝐡𝐢𝐧𝐤𝐢𝐧𝐠 𝐒𝐭𝐫𝐚𝐭𝐞𝐠𝐢𝐜𝐚𝐥𝐥𝐲

Greedy search is short-sighted; Beam Search is a strategist. It explores multiple paths (the Beam Width) at once, keeping the top "N" sequences that have the highest cumulative probability over time.

🔹 𝐅𝐫𝐨𝐦 𝐭𝐡𝐞 𝐢𝐦𝐚𝐠𝐞: The model tracks candidates through multiple iterations, pruning weak paths and keeping the strongest "beams."

🔹 𝐁𝐞𝐬𝐭 𝐟𝐨𝐫: Tasks where long-term coherence is more important than the immediate next word.

𝟒. 𝐂𝐨𝐧𝐭𝐫𝐚𝐬𝐭𝐢𝐯𝐞 𝐒𝐞𝐚𝐫𝐜𝐡: 𝐅𝐢𝐠𝐡𝐭𝐢𝐧𝐠 𝐑𝐞𝐩𝐞𝐭𝐢𝐭𝐢𝐨𝐧

A common flaw in AI is "looping." Contrastive search solves this by penalizing tokens that are too similar to what was already written using Cosine Similarity.

🔹 𝐅𝐫𝐨𝐦 𝐭𝐡𝐞 𝐢𝐦𝐚𝐠𝐞: It takes the top-k tokens (k=4) and subtracts a "Penalty." Even if a word has high probability, it might be skipped if it's too repetitive, allowing a word like "set" to be chosen instead.

🔹 𝐁𝐞𝐬𝐭 𝐟𝐨𝐫: Long-form content and maintaining a natural "flow."

💡 𝐓𝐡𝐞 𝐓𝐚𝐤𝐞𝐚𝐰𝐚𝐲:

There is no single "best" way to generate text. Most AI applications today use a blend of these strategies to balance accuracy with human-like variety.

𝗪𝐡𝐢𝐜𝐡 𝐬𝐭𝐫𝐚𝐭𝐞𝐠𝐲 𝐝𝐨 𝐲𝐨𝐮 𝐭𝐡𝐢𝐧𝐤 𝐩𝐫𝐨𝐝𝐮𝐜𝐞𝐬 𝐭𝐡𝐞 𝐦𝐨𝐬𝐭 "𝐡𝐮𝐦𝐚𝐧" 𝐫𝐞𝐬𝐮𝐥𝐭𝐬? 𝐋𝐞𝐭’𝐬 𝐝𝐢𝐬𝐜𝐮𝐬𝐬 𝐢𝐧 𝐭𝐡𝐞 𝐜𝐨𝐦𝐦𝐞𝐧𝐭𝐬! 👇

#GenerativeAI #LLM #MachineLearning #NLP #DataScience #AIEngineering


r/neuralnetworks • • Sep 26 '26

Neural network from scratch in NumPy with an app to watch it learn and edit single neurons :)

Thumbnail
gallery
189 Upvotes

Hi everyone, I made this to understand backprop for real and I think it can be useful to others who are learning.

It's a small neural network written by hand in NumPy (no PyTorch, no autograd) that learns to read handwritten digits from MNIST, with a desktop app that shows what happens inside while it trains. The whole network is one file of about 140 lines, and there's a test that checks the backprop against the numerical gradient.

Some things you can do with it:

• watch the gradient norm of each layer and the % of inactive neurons while it trains, and change learning rate or dropout without stopping it

• see what each neuron of the first layer "looks for" and how the weights change compared with how they started

• switch off or rescale single neurons, prune, add noise to the weights and see the test accuracy change right away

• follow the math of a layer cell by cell, with the softmax step by step

With all the 60,000 MNIST photos it gets to about 98.5%, on the CPU.

To try it pip install neural-network-digits, or download the zip from the releases. Most of the code was written with Claude Code (AI), the idea and what to show in every tab are mine. It's MIT.

https://github.com/dev-luigi/neural-network-digits

What would you add to make it more useful for someone who is learning?


r/neuralnetworks • • Nov 18 '20

I work with models

Post image
172 Upvotes

r/neuralnetworks • • Mar 17 '20

Monna Lisa upscaled, this is not photoshop

Post image
156 Upvotes

r/neuralnetworks • • Jun 04 '22

My neural network from scratch is finally doing aomething :)

Post image
143 Upvotes

r/neuralnetworks • • Feb 20 '21

Perceptron Neural Network Visualized Interpreting Drawn Numbers

142 Upvotes

r/neuralnetworks • • 6d ago

Need partner

Post image
136 Upvotes

Hi,I read this book these days

I really wanna some one who read it as well ,to share some idea or even help each other I'm at regularization chapter

So,if you are interested pls DM

And thanks 😊


r/neuralnetworks • • Nov 19 '19

First personal project. I took the classic game of Snake and gave it a genetic algorithm.

Thumbnail
i.imgur.com
134 Upvotes

r/neuralnetworks • • 23d ago

I JUST MADE MY FIRST NEURAL NETWORK!!

Post image
129 Upvotes

After lots of research and reading over the last few days, I made my first working neural network. I just programmed it in JavaScript during work (when I wasn't working ;) and it works!!!

The inputs are a 2x2 grid, and the outputs are different patterns on the grid (checkerboard, stairs, upside-down stairs, and alone). Pretty awesome stuff!!

The screenshot is from the HTML page I made for the visuals, nothing crazy, but fun to look at.


r/neuralnetworks • • Dec 06 '25

Flappy Flappy Flying RIght, In the Pipescape of the Night

121 Upvotes

Wanted to share this with the community. It is just flappy bird but it seems to learn fast using a pipeline of evolving hyperparameters along a vector in a high dimensional graph, followed by short training runs and finally developing weights of "experts" in longer training. I have found liquid nets fascinating, lifelike but chaotic - so finding the sweet spot for maximal effective learning is tricky. (graph at bottom attempts to represent hyperparameter fitness space.) It is a small single file and you can run it: https://github.com/DormantOne/liquidflappy This applies the same strategy we have used for our falling brick demo, but since it is a little bit harder introduces the step of selecting and training early performance leaders. I keep thinking of that old 1800s Blake poem Tyger Tyger Burning Bright In the Forest of the Night - the line "in what furnace was thy brain?" seems also the question of modern times.


r/neuralnetworks • • Feb 20 '26

I’m trying to understand this simple neural network equation:

Post image
111 Upvotes

My questions:

  1. Why do we use XT W instead of WX?

  2. Is this representing a single neuron in a neural network?

I understand basic matrix multiplication, but I want to make sure I’m interpreting this correctly.


r/neuralnetworks • • Jan 27 '26

I made a Python library for Graph Neural Networks (GNNs) on geospatial data

Thumbnail
gallery
108 Upvotes

I'd like to introduce City2Graph, a new Python package that bridges the gap between geospatial data and graph-based machine learning.

What it does:

City2Graph converts geospatial datasets into graph representations with seamless integration across GeoPandas, NetworkX, and PyTorch Geometric. Whether you're doing spatial network analysis or building Graph Neural Networks for GeoAI applications, it provides a unified workflow:

Key features:

  • Morphological graphs: Model relationships between buildings, streets, and urban spaces

  • Transportation networks: Process GTFS transit data into multimodal graphs

  • Mobility flows: Construct graphs from OD matrices and mobility flow data

  • Proximity graphs: Construct graphs based on distance or adjacency

Links:


r/neuralnetworks • • Feb 16 '21

I resurrected Albert Einstein (1879-1955) at age of 8 with Artificial Intelligence.

Post image
108 Upvotes

r/neuralnetworks • • 12d ago

How a neural network learns - a simplified and visualized explainer app

105 Upvotes

I recently built and published an explainer on visualising how a convolutional digit-recognition neural network "thinks" in real time.

My kids loved it and asked me to build one on how it LEARNS.
This new explainer does exactly that:

  • Take the interactive and visual tour to explore the learning process step by step

  • Experience 3 different types of learning (Supervised, Imitation, Reinforcement Learning)

  • See how the model "learns to drive", challenge the learning process to see what makes it fail or succeed.

As with the previous demo, my kids loved it!

Share it with yours, or with anyone genuinely new to, and curious about, Machine Learning.