r/knowm • • Nov 25 '15

Stability of AHaH synapses (simulation)

0 Upvotes

Stability of AHaH synapses (simulation)

Alex has invited me several times to post my simulations of kT-RAM. I've decided to accept that invitation and do it in parts, starting with this simulation study of AHaH synapse memory stability.

Summary: The kT-RAM architecture requires destructive reads of the memristors in its AHaH synapses. Alex claims that the resulting damage to stored synapse memories can be repaired by "anti-Hebbian" read operations. I show through simulation that this is not the case, and explain why nonlinearies in memristor dynamics preclude this from working.

Introduction

If you run into some old friends you haven't seen in a few years, you can still recognize them. If you go on a long camping trip without any books, you can still read when you get back. Stability of long-term memories without continual reinforcement from the environment is essential for survival. It's also a practical requirement for a deployed machine learning system.

Knowm's kT-RAM is built on memristive devices for storing analog memories. The kT-RAM architecture requires that those memristive devices be read destructively:

There is no such thing as a 'non-destructive read' operation in kT-RAM. Every memory access results in weight adaptation...

The destructive reads require additional repair processes, but the payoff is claimed to be worth it:

If you can repair this constant damage, you get the low-power adaptive learning solution... and...a scaling path to much higher levels of adaptive efficiency.

Part of this repair is done by ongoing reinforcement from the environment. But if that reinforcement is intermittent (e.g. a vision system that was trained to read text that then goes on an extended camping trip with no books), additional repair processes are needed.

On this thread, Alex describes such a process and claims:

The combination of Anti-Hebbian and Hebbian read operations appears sufficient to repair damage to synaptic states as they are used and we have demonstrated this over multiple benchmark datasets and multiple memristor models.

Let's investigate this claim with a simulation.

Simulating an AHaH synapse

An AHaH node (the kT-RAM analog of a neuron) is

...built up from one or more synapses, which are implemented with differential memristor pairs.

So our simulation is of the simplest case: an AHaH node with a single synapse consisting of two memristors. We'll train it, then "send it on a camping trip" where it's unable to get supervision or restorative reinforcement from the environment. But since it's part of a larger system that must remain functioning, it continues to be read repetitively. Its repair must rely on the Hebbian / Anti-Hebbian read algorithm (the FF and RF kT-RAM instructions) from Figure 3 of the Cortical Processing paper.

The simulation was written in Python based on the AHaH Computing paper. Two simple memristor models were implemented, the linear (LID) and nonlinear (NLID)ion drift models. The learning rates of the two models were tweaked to be roughly equivalent. Voltages and pulse widths were those used in the AHaH computing paper. The state variable for each memristor, w, was initialized to the middle of its range. The synapse was "trained" for 25 cycles, then read repeatedly using the Hebbian / Anti-Hebbian algorithm suggested by Alex.

The following table shows the temporal evolution of the synapse memory, which is read from the voltage at the junction of the two memristors

AHaH Stability Experiment Results

cycleLID memristorsNLID memristors
00.300050.30005
10.299860.29998
20.299670.29992
30.299490.29985
40.299300.29978
50.299120.29972
60.298940.29965
70.298750.29958
80.298570.29952
90.298390.29945
100.298200.29938
110.298020.29932
120.297840.29925
130.297660.29918
140.297480.29912
150.297300.29905
160.297130.29898
170.296950.29891
180.296770.29884
190.296590.29878
200.296420.29871
210.296240.29864
220.296070.29857
230.295890.29850
240.295720.29843
250.295540.29836
10,0000.133490.16756
50,0000.061460.07899
500,0000.000000.00000

In the above table, the first columns shows the cycle number of the simulation, and the next two columns show the "memory" (memristor junction voltage) in the two different versions of the synapse. It's clear that there is a very slow decay of the memory, roughly 1% of the rate at which learning took place. After 500,000 read cycles, the memory is gone.

Source of the memory decay

The primary source of the memory decay is nonlinearity in the dynamics of the memristors. A memristor's conductance is bounded below (it can't become negative) and above (it can't become infinite, which would imply zero resistance). As a memristor is "spiked," its change in conductance slows down as it approaches a boundary, becoming zero right at the boundary. But if the polarity of the spike is then reversed, the memristor can change conductance as it moves away from the boundary. Notice the asymmetry near the boundary: a step towards the boundary has little affect on conductance, while a step away from the boundary does.

This asymmetry causes problems. Alex's Hebbian / Anti-Hebbian read cycle takes one step towards a boundary and one step back. But the step sizes are slightly different for reasons just discussed, so one step does not completely cancel out the other. As seen in the experiment, this creates a weak attractor in the memristor configuration that slowly decays the memory. The attractor is weak, and the decay is slow. But it is relentless.

(The situation is slightly more complex than just described since one has to consider the interacting dynamics of the two memristors. But those affects are secondary.)

Stabilizing volatile analog memories is hard

Attractors have long been used to stabilize volatile memories. An attractor is just a process that monitors degradation of a memory and "pulls it back" to where it's supposed to be when it wanders away too far.

For example, a latch in a digital circuit uses feedback to stabilize its output voltage to one of two attractor voltages--voltages representing logical "1" and logical "0". If that output voltage starts to wander away from the attractor, either due to noise or general insubordination, the feedback acts like a cattle prod and gooses it back.

DRAM uses a different mechanism to stabilize it's highly unstable bits: refresh. Each memory bit is periodically read (before it's had a chance to sag too much) and written back in a robust state.

The above attractor examples were for bits, but attractors can also work on a block of bits. An error-correcting code (ECC) does this: if a bit drops out, ECC returns the value of the nearest attractor rather than the corrupted block value.

Attractors for analog memories are much more challenging because in order to repair a decayed analog memory, you need to know what the value of the memory was before the decay (or, equivalently, the precise value of the decay). There's no mechanism in kT-RAM for getting that information, so kT-RAM is unable to implement an attractor to stabilize the volatile memory. The best it can do is slow it down with it's one-step-forward, one-step-back read cycle. That is an open loop process, and the memory decays and wanders away as shown in the simulation.

So why did Alex's experiments show it was stable?

Let's go back to Alex's statement near the top of this post with some added emphasis:

The combination of Anti-Hebbian and Hebbian read operations appears sufficient [italics mine] to repair damage to synaptic states as they are used and we have demonstrated this over multiple benchmark datasets and multiple memristor models.

The "appears sufficient" phrase strongly suggests that Alex is aware that there is no attractor stabilization in his AHaH memory. He is appealing to empirical results. So why did his results show the one-step-forward, one-step-back is sufficient? Since I don't have access to his test cases, I can only do informed speculation based on his publications. There are two possibilities.

The first is that Alex ran his benchmarks with the usual training pass followed by a test pass. That is appropriate when running on a conventional processor, but can be misleading for an adaptive system with destructive reads like kT-RAM. A better test would be to: (1) train the system; (2) send the system on vacation for a long time (by exposing it to inputs not seen in the training set--for example, exposing a handwritten digit classifier to pictures of trees and birds); (3) then test the system.

The second possibility is that the kT-RAM functional model is not an accurate approximation of the circuit model. For example, if Alex simply assumed that the Hebbian / Anti-Hebbian read cycle was stable, he might have hard-coded that into his functional model.

Discussion

The kT-RAM architecture requires destructive reads of its memristors. Repairing the damage uses an open-loop, one-step-forward / one-step-back process that slows down the degradation but cannot eliminate it because of nonlinearities in the memristor dynamics. Attractor stabilization of memristor memories is not possible in kT-RAM as currently designed. Thus kT-RAM will not be suitable for machine learning tasks requiring stable long-term memory in an environment that can only offer intermittent reinforcement.


r/knowm • • Apr 22 '15

[PDF] Bio-inspired Unsupervised Learning of Visual Features Leads to Robust Invariant Object Recognition

Thumbnail arxiv.org
0 Upvotes

r/knowm • • Jul 01 '15

Nonlinear Hebbian learning as a universal principle in unsupervised feature learning

Thumbnail 8109f4a4-a-62cb3a1a-s-sites.googlegroups.com
0 Upvotes

r/knowm • • Jun 15 '16

Neuronal arithmetic

Thumbnail
nature.com
1 Upvotes

r/knowm • • Jul 07 '15

Knowm.org | Knowm Development Community

Thumbnail knowm.org
2 Upvotes

r/knowm • • Nov 17 '15

The Problem with "The Adaptive Power Problem"

2 Upvotes

Review: The Problem with the "Adaptive Power Problem"

Short version: Alex Nugent, CEO of Knowm, claims to have discovered a new principle of computation found throughout nature ("In Nature's Computers, d = 0") which will allow us to design computers that are up to 10 billion times more efficient than existing computers. Unfortunately his thought experiment illustrating this principle has a fatal flaw which, when corrected, turns the thought experiment into a counterexample. He also used this principle to design a version of kT-RAM, about which he now admits "Capacitive losses...would be very high...and throughput would be low", thus creating a second counterexample. The reason: there is no such principle.

What is the "Adaptive Power Problem"?

Brains are much more energy efficient than digital computers at solving some (but not all) classes of problems. Researchers have pondered this for decades, but Alex thinks he understands why: unlike human-designed computers, brains do their computation using memory and processing in "the same place." They don't separate them using the von Neumann computation model. Having them "in the same place" eliminates the "shuttling of information back and forth" between processor and memory, thus eliminating the vast bulk of the capacitive energy losses that plague modern computer architectures. Once we realize this (and apparently no other computer architects have) we can design architectures like kT-RAM that have "power efficiency gains of up to 10 orders of magnitude over traditional computing architectures" and are "hundreds to thousands of times" more efficient than future competitors yet still deliver near state-of-the-art performance on machine learning problems.

In his description of the Adaptive Power Problem, Alex says:

"Based on the known laws of Physics and our insistence on the separation of memory and processing, it is not possible to simulate biology at biological efficiency."

Aha. Since we can't do anything about the laws of Physics, Alex is saying that the problem must lie in the "separation of memory and processing." This is where "d = 0" comes from--d is the distance between memory and processor.

He illustrates this with a thought experiment of simulating a human body in great detail using a hypothetical von Neumann mesh supercomputer. Although he doesn't state this explicitly, his human body model appears to be a dynamical system comprising an enormous number of state variables (5,000,000,000,000,000) plus the corresponding differential equations that model the interactions between those variables. The interactions are predominantly local. Running the simulation involves numerically integrating those differential equations on the supercomputer.

Using some simplifying assumptions, he estimates that the power dissipated in the wires between the processors and memory units for his simulation to be 160 trillion watts. That's a lot of power, and presumably he derived this big number to illustrate the Adaptive Power Problem. But what he hasn't done is estimate the power dissipated by either the processors or memory systems. And therein lies the real problem.

Let's continue Alex's thought experiment and estimate processor power. Keeping it simple like Alex did, let's assume each differential equation depends on only 50 state variables, all available locally. We'll also ignore all overhead in the CPU for dispatching the computation. Integrating one time step for each differential equation thus requires at least 100 FLOPS (floating point operations), each of which will consume, say, 320 pJ. Thus each state variable will require ~32,000 pJ of processing energy each timestep to compute its next state. Writing that new state out to memory will take (using Alex's equations) about 32 pJ. So that allows us to make a rough estimate:

  • Power in wires: 160 trillion watts

  • Power in processors: 160,000 trillion watts !

In other words, the power dissipated in the wires doesn't matter at all! Total system power is completely dominated by the processors. The power in the wires is only ~ 0.1% of the power dissipated by the system.

(You may now slap yourself on the forehead and mumble "So what was the point of Alex's thought experiment?")

Like everyone else in the industry, Alex knows that capacitive wire losses are throttling performance gains in digital computers. But he jumps to the conclusion that it's the wires between memory and processor in the von Neumann architecture that are the guilty parties. As shown above, this is not necessarily the case. This mistake is what led Alex to the false conclusion that putting memory and processing in "the same place" solves this problem. It doesn't. He simply picked the wrong set of wires. (See the Peter Kogge article to read about the real culprits.)

"In Nature's Computer, D=0"

Alex's equation for estimating the capacitive wire losses contains a scale factor called d which represents the distance between processor and memory. In the human body simulation he set d to one cm. But if you can make d equal to zero, then the wire power also goes to zero. Pretty cool, huh? This seems to be the core philosophical insight that has led him astray:

"But recognize that your assumptions place real physical bounds on what you can, and can't, do. Most importantly of all, you should recognize that in all brains, all life, and indeed everywhere in Nature outside of our computers, d is zero."

That's an eloquent little piece of writing. But even if it were true (it's not), setting d to zero had no impact on system power in Alex's thought experiment. The wire energy is negligible in comparison. (Energy dissipation in wires does matter--a lot--but not for the reasons he described.)

Memory and Processing in the Same Place

Well, something went wrong. So let's switch to biology--surely putting memory and processing "in the same place" there will work. Otherwise, how could brains be so energy efficient?

A neuron is biological, so does it do memory and processing in the same place? According to Alex, it does:

A neuron does not separate memory and processing and shuttle bits back and forth. It is a merging of memory and processing.

A synapse is not memory and its not processing--its a merging of the two.

A soma is not memory and its not processing. Its a merging of the two.

Most neurobiologists would be a little uncomfortable with the semantic gamesmanship in that statement. It's generally believed that synapses hold long-term memories, while state variables in the soma work on shorter time scales, for example to manage homeostasis, integrate incoming weighted spikes, and generate output spikes.

Sure, some processing goes on in the synapses, and there are state variables (memory) in the soma. But the exact same thing is true in a computer. Memory systems contain a lot of processing: refresh, error detection and correction, wear leveling, etc. And processors have a lot of state: flipflops, register files, caches. Both subsystems have "merged processing and memory."

Let me take Alex's passage above and make a couple of italisized substitutions to make this clearer:

A processor is not memory and its not processing--its a merging of the two.

A *memory bank" is not memory and its not processing. Its a merging of the two.

Alex's black-and-white segmentation of neurons into the "completely merged" memory and processing category, and conventional computers into the "completely separated" memory and processing category, is arbitrary. Worse that that, it's just wrong. There is no objective criterion for that dichotomy. He makes that distinction only to support his thesis.

There are just way more losses in a digital computer trying to calculate than in the real thing.

If you're trying to simulate a brain, I agree. But for reasons completely unrelated to the merging of memory and processing. (That is an interesting topic on its own.)

Incidentally, there are many domains where computers are way more efficient than human brains. Would you care to integrate 131,072 coupled differential equations to implement a quantum simulation in your head? A laptop could do that in minutes for pennies of electricity. A brain wouldn't be able to finish that in its lifetime.

Alex goes on to say:

It means that calculation of very large numbers of interacting adaptive variables via the separation of memory and processing is overwhelming less efficient

That is precisely what he didn't show. When I modified his hypothetical mesh supercomputer so that memory and processing were magically merged somehow, d would be zero. Big deal. That reduces system power by only 0.1% because you still have to do all the computation to update the state variables. Alex's original supercomputer was not "overwhelming less efficient"--it was negligibly less efficient for his problem. Read this to see where the problems of evaluating large numbers of interacting adaptive variables really lie.

At least kT-RAM is super-efficient. Right?

According to Alex, it should be. After all, it mimics the structure of energy-efficient neurons: AHaH nodes (memristor pairs) correspond to synapses, and the H-tree wiring and comparator correspond to the soma. And most importantly, one has to assume it obeys the computational principle "d = 0" he discovered while pondering the Adaptive Power Problem. What's the point of discovering a new computational principle if you don't apply it?

There is a long discussion of this here and here. But the bottom line is that Alex admitted that a specific instance of kT-RAM he proposed in his paper (see Figure 4 and section II.D), was not efficient: "Capacitive losses in kT-RAM would be very high in this case, and throughput would be low." His "d = 0" principle failed him for some reason. (See the above links for details.)

So he tried to recover by saying kT-RAM cores should be tiny, embedded in a routing mesh. EEs will immediately see that doing so will transfer some of the capacitive losses in the kT-RAM H-tree to the wires in the routing mesh. How efficient would that architecture be? Apparently Alex doesn't know, or at least is unwilling to say, because he asked me to simulate it for him.

I guess this new computational principle, "d = 0", found in nature is just fickle. It sneaks away when you need it the most.

Conclusions

The Adaptive Power Problem and the resulting "d = 0" design principle are red herrings.

There is no clean separation of memory and processing in computers as Alex claims. A processor is a complex, tangled quilt of memory (flip-flops, register files, caches) computation (ALU) and control circuitry. The same holds true for memory systems. His thought experiment for demonstrating "d = 0" turned out to be a counterexample. His failed kT-RAM design, also using the "d = 0" principle, is a second counterexample.

A wire doesn't care if it's carrying a signal in a memory module as opposed to a processor module. Wire capacitance is wire capacitance. Minimizing capacitive losses requires good architectural choices (e.g. caches, layout, interconnect), and careful implementation. This is true regardless of whether the underlying computation is digital or analog. Merging memory and processing as demanded by his "d = 0" principle is simply not a requirement.


r/knowm • • Jul 06 '15

NP-complete Problems and Physical Reality by Scott Aaronson

Thumbnail scottaaronson.com
0 Upvotes

r/knowm • • Jun 01 '18

Prediction Machines: The Simple Economics of Artificial Intelligence

Thumbnail
youtube.com
0 Upvotes

r/knowm • • May 07 '15

Training and operation of an integrated neuromorphic network based on metal-oxide memristors : Nature

Thumbnail nature.com
1 Upvotes

r/knowm • • May 13 '15

PolyStat: Of programming for machine learning, speed and Julia (also Java is fast)

Thumbnail
polystat.blogspot.ca
1 Upvotes

r/knowm • • Sep 17 '15

Nine Post-Moore Technologies

Thumbnail
datascienceassn.org
1 Upvotes

r/knowm • • Sep 16 '15

Selecting a camera and lens for your robot

Thumbnail
robohub.org
1 Upvotes

r/knowm • • Jul 25 '17

37 Reasons why your Neural Network is not working

Thumbnail
blog.slavv.com
1 Upvotes

r/knowm • • Jul 11 '15

Knowms from space

Thumbnail
atlasobscura.com
1 Upvotes

r/knowm • • Jul 07 '15

Grid Long Short-Term Memory

Thumbnail
arxiv.org
1 Upvotes

r/knowm • • Nov 17 '15

The quantum source of space-time

Thumbnail
nature.com
1 Upvotes

r/knowm • • Mar 27 '15

Specialization and natural learning.

1 Upvotes

Within the beginning of the system I have been led to believe the memristors would be neutrally aligned between their two outputs. Once a signal has been sent through the channel it would favor either a or b. It seems like the best way to facilitate an increase in either a or b would be to decrease the resistance to the next node. This would then branch further into the system until there is a path of least resistance. I believe I have a misunderstanding, but I don't know where.

The neutral system would potentially be capable of natural learning for any reasonably limited task. Once the system has evolved into an efficient machine it would gain specialization at the loss of learning ability. You could teach the system to do something, but in doing so it would lose it's ability to learn a different task. Assuming a limited number of Kt-RAM.

So would each system fall into specialization limited by it's resources, or would the system be more fluid in it's intelligence?


r/knowm • • Apr 22 '15

"...visual attention is being driven by the category of the target and not by specific visual features of that target."

Thumbnail
psychologytoday.com
1 Upvotes

r/knowm • • Jun 27 '16

Precise deep neural network computation on imprecise low-power analog hardware

Thumbnail
arxiv.org
2 Upvotes

r/knowm • • Jul 06 '16

Machine Learning Mastery With Weka

Thumbnail
machinelearningmastery.com
2 Upvotes

r/knowm • • Apr 19 '15

Knowm.org | Video: Introduction to AHaH Computing Filmed April 9th 2015 at Rochester Institute of Technology

Thumbnail
knowm.org
2 Upvotes

r/knowm • • Apr 28 '15

Kickstarter Fantastic Fungi Documentary on Mycelium

Thumbnail
vimeo.com
2 Upvotes

r/knowm • • Sep 21 '15

[DeepMind Paper 2014] Recurrent Models of Visual Attention

Thumbnail arxiv.org
2 Upvotes

r/knowm • • Sep 27 '15

Other memelements than the memristor.

2 Upvotes

It appears the memristor is the main focus of Knowm. The memristor has been studied significantly more than other memelements. These other elements include the memcapacitor and meminductor. I was wondering how much interest Knowm had in exploring these other elements.


r/knowm • • Sep 23 '15

How do you spike a node?

2 Upvotes

So I've come to understand that memristors change their resistance after a voltage pulse. A high voltage pulse increases the resistance and a negative voltage pulse decreases resistance.

How exactly do you spike a node? Specifically how do you target an individual memristor instead of causing a current through the whole circuit?

I understand how spike encoding works with 1 as a positive pulse and 0 as a negative pulse, but I'm not sure I understand how it actually would work in practice. I guess I'm trying to think of it in discrete logic terms and in basic circuit terms.

Can memristors be explained using KCL, KVL, Ohm's law and such basic concepts?

Is there any formulation for the logic behind AHah computing?

A two input AHaH node will receive three possible spike patterns {(1,z),(z,1),(1,1)} and converge to multiple attractor states.

How do you use this to create logic statements?

I can understand how the universal logic can be created by NAND and XOR gates. I don't know what memristors would do differently though.