r/knowm • u/Gordon-Panthana • Nov 25 '15
Stability of AHaH synapses (simulation)
Stability of AHaH synapses (simulation)
Alex has invited me several times to post my simulations of kT-RAM. I've decided to accept that invitation and do it in parts, starting with this simulation study of AHaH synapse memory stability.
Summary: The kT-RAM architecture requires destructive reads of the memristors in its AHaH synapses. Alex claims that the resulting damage to stored synapse memories can be repaired by "anti-Hebbian" read operations. I show through simulation that this is not the case, and explain why nonlinearies in memristor dynamics preclude this from working.
Introduction
If you run into some old friends you haven't seen in a few years, you can still recognize them. If you go on a long camping trip without any books, you can still read when you get back. Stability of long-term memories without continual reinforcement from the environment is essential for survival. It's also a practical requirement for a deployed machine learning system.
Knowm's kT-RAM is built on memristive devices for storing analog memories. The kT-RAM architecture requires that those memristive devices be read destructively:
There is no such thing as a 'non-destructive read' operation in kT-RAM. Every memory access results in weight adaptation...
The destructive reads require additional repair processes, but the payoff is claimed to be worth it:
If you can repair this constant damage, you get the low-power adaptive learning solution... and...a scaling path to much higher levels of adaptive efficiency.
Part of this repair is done by ongoing reinforcement from the environment. But if that reinforcement is intermittent (e.g. a vision system that was trained to read text that then goes on an extended camping trip with no books), additional repair processes are needed.
On this thread, Alex describes such a process and claims:
The combination of Anti-Hebbian and Hebbian read operations appears sufficient to repair damage to synaptic states as they are used and we have demonstrated this over multiple benchmark datasets and multiple memristor models.
Let's investigate this claim with a simulation.
Simulating an AHaH synapse
An AHaH node (the kT-RAM analog of a neuron) is
...built up from one or more synapses, which are implemented with differential memristor pairs.
So our simulation is of the simplest case: an AHaH node with a single synapse consisting of two memristors. We'll train it, then "send it on a camping trip" where it's unable to get supervision or restorative reinforcement from the environment. But since it's part of a larger system that must remain functioning, it continues to be read repetitively. Its repair must rely on the Hebbian / Anti-Hebbian read algorithm (the FF and RF kT-RAM instructions) from Figure 3 of the Cortical Processing paper.
The simulation was written in Python based on the AHaH Computing paper. Two simple memristor models were implemented, the linear (LID) and nonlinear (NLID)ion drift models. The learning rates of the two models were tweaked to be roughly equivalent. Voltages and pulse widths were those used in the AHaH computing paper. The state variable for each memristor, w, was initialized to the middle of its range. The synapse was "trained" for 25 cycles, then read repeatedly using the Hebbian / Anti-Hebbian algorithm suggested by Alex.
The following table shows the temporal evolution of the synapse memory, which is read from the voltage at the junction of the two memristors
AHaH Stability Experiment Results
| cycle | LID memristors | NLID memristors |
|---|---|---|
| 0 | 0.30005 | 0.30005 |
| 1 | 0.29986 | 0.29998 |
| 2 | 0.29967 | 0.29992 |
| 3 | 0.29949 | 0.29985 |
| 4 | 0.29930 | 0.29978 |
| 5 | 0.29912 | 0.29972 |
| 6 | 0.29894 | 0.29965 |
| 7 | 0.29875 | 0.29958 |
| 8 | 0.29857 | 0.29952 |
| 9 | 0.29839 | 0.29945 |
| 10 | 0.29820 | 0.29938 |
| 11 | 0.29802 | 0.29932 |
| 12 | 0.29784 | 0.29925 |
| 13 | 0.29766 | 0.29918 |
| 14 | 0.29748 | 0.29912 |
| 15 | 0.29730 | 0.29905 |
| 16 | 0.29713 | 0.29898 |
| 17 | 0.29695 | 0.29891 |
| 18 | 0.29677 | 0.29884 |
| 19 | 0.29659 | 0.29878 |
| 20 | 0.29642 | 0.29871 |
| 21 | 0.29624 | 0.29864 |
| 22 | 0.29607 | 0.29857 |
| 23 | 0.29589 | 0.29850 |
| 24 | 0.29572 | 0.29843 |
| 25 | 0.29554 | 0.29836 |
| 10,000 | 0.13349 | 0.16756 |
| 50,000 | 0.06146 | 0.07899 |
| 500,000 | 0.00000 | 0.00000 |
In the above table, the first columns shows the cycle number of the simulation, and the next two columns show the "memory" (memristor junction voltage) in the two different versions of the synapse. It's clear that there is a very slow decay of the memory, roughly 1% of the rate at which learning took place. After 500,000 read cycles, the memory is gone.
Source of the memory decay
The primary source of the memory decay is nonlinearity in the dynamics of the memristors. A memristor's conductance is bounded below (it can't become negative) and above (it can't become infinite, which would imply zero resistance). As a memristor is "spiked," its change in conductance slows down as it approaches a boundary, becoming zero right at the boundary. But if the polarity of the spike is then reversed, the memristor can change conductance as it moves away from the boundary. Notice the asymmetry near the boundary: a step towards the boundary has little affect on conductance, while a step away from the boundary does.
This asymmetry causes problems. Alex's Hebbian / Anti-Hebbian read cycle takes one step towards a boundary and one step back. But the step sizes are slightly different for reasons just discussed, so one step does not completely cancel out the other. As seen in the experiment, this creates a weak attractor in the memristor configuration that slowly decays the memory. The attractor is weak, and the decay is slow. But it is relentless.
(The situation is slightly more complex than just described since one has to consider the interacting dynamics of the two memristors. But those affects are secondary.)
Stabilizing volatile analog memories is hard
Attractors have long been used to stabilize volatile memories. An attractor is just a process that monitors degradation of a memory and "pulls it back" to where it's supposed to be when it wanders away too far.
For example, a latch in a digital circuit uses feedback to stabilize its output voltage to one of two attractor voltages--voltages representing logical "1" and logical "0". If that output voltage starts to wander away from the attractor, either due to noise or general insubordination, the feedback acts like a cattle prod and gooses it back.
DRAM uses a different mechanism to stabilize it's highly unstable bits: refresh. Each memory bit is periodically read (before it's had a chance to sag too much) and written back in a robust state.
The above attractor examples were for bits, but attractors can also work on a block of bits. An error-correcting code (ECC) does this: if a bit drops out, ECC returns the value of the nearest attractor rather than the corrupted block value.
Attractors for analog memories are much more challenging because in order to repair a decayed analog memory, you need to know what the value of the memory was before the decay (or, equivalently, the precise value of the decay). There's no mechanism in kT-RAM for getting that information, so kT-RAM is unable to implement an attractor to stabilize the volatile memory. The best it can do is slow it down with it's one-step-forward, one-step-back read cycle. That is an open loop process, and the memory decays and wanders away as shown in the simulation.
So why did Alex's experiments show it was stable?
Let's go back to Alex's statement near the top of this post with some added emphasis:
The combination of Anti-Hebbian and Hebbian read operations appears sufficient [italics mine] to repair damage to synaptic states as they are used and we have demonstrated this over multiple benchmark datasets and multiple memristor models.
The "appears sufficient" phrase strongly suggests that Alex is aware that there is no attractor stabilization in his AHaH memory. He is appealing to empirical results. So why did his results show the one-step-forward, one-step-back is sufficient? Since I don't have access to his test cases, I can only do informed speculation based on his publications. There are two possibilities.
The first is that Alex ran his benchmarks with the usual training pass followed by a test pass. That is appropriate when running on a conventional processor, but can be misleading for an adaptive system with destructive reads like kT-RAM. A better test would be to: (1) train the system; (2) send the system on vacation for a long time (by exposing it to inputs not seen in the training set--for example, exposing a handwritten digit classifier to pictures of trees and birds); (3) then test the system.
The second possibility is that the kT-RAM functional model is not an accurate approximation of the circuit model. For example, if Alex simply assumed that the Hebbian / Anti-Hebbian read cycle was stable, he might have hard-coded that into his functional model.
Discussion
The kT-RAM architecture requires destructive reads of its memristors. Repairing the damage uses an open-loop, one-step-forward / one-step-back process that slows down the degradation but cannot eliminate it because of nonlinearities in the memristor dynamics. Attractor stabilization of memristor memories is not possible in kT-RAM as currently designed. Thus kT-RAM will not be suitable for machine learning tasks requiring stable long-term memory in an environment that can only offer intermittent reinforcement.