Downside risk

Sparse distributed memory is a mathematical model of human long-term memory introduced by Pentti Kanerva in 1988. It is used for storing and retrieving large amounts ($2^{1000}$ bits) of information without focusing on the accuracy of the information. It uses patterns to serve as memory addresses, where information is retrieved based on similarities between addresses. Memory addresses are all in a list even if they are not related, and are retrieved based on similar content.

Formula

The general formula is $2^{n}$ where n is the number of dimensions of the space, and $2^{n}$ is the number of feasible memory items.

Critical Distance

The critical distance of a Sparse Distributed Memory can be approximately evaluated minimizing the following equation with the restriction $d\in N$ and $d\leqslant n$. The proof can be found in

${\tilde {f}}(d)=\left\{{\frac {1}{2}}\cdot \left[1-N\left(z<{\frac {w\cdot shared(d)}{\sqrt {\theta }}}\right)+N\left(z<{\frac {-w\cdot shared(d)}{\sqrt {\theta }}}\right)\right]-{\frac {d}{n}}\right\}^{2}$

Where:

• $d$: be the distance to the target;
• $h$: be the number of hard-locations activated during read and write operations (this value depends on access radius values);
• $s$: be the number of total stored bitstrings in memory;
• $H$: be the number of hard-locations in memory;
• $w$: be the number of times the target bitstring was written in memory;
• $\theta$: be the total of random bitstrings in all $h$ hard-locations activated by a read operation;
• $shared(d)$: be the mean number of shared hard-locations activated by two bitstrings $d$ bits away from each other. One can find some values for a 1000-dimensional SDM in Kanerva's book, Table 7.1, p. 63, or the equations to calculate to any SDM in Appendix B, p. 125 of the same book.

Definition

Principle

Sparse distributed memory is a mathematical representation of human memory, and uses high-dimensional space to help model the large amounts of memory that mimics that of the human neural network. It utilizes the Hamming distance to measure mismatched bits and read back data between the original write address and one near it. Human memory has a tendency to congregate memories based on similarities between them(although they may not be related), such as "firetrucks are red and apples are red".

Neurons

Neurons are electrically excitable cells that transmit information to and from the brain. They are used as models for sending and retrieving data in a sparse distributed memory system. Neurons are the cells that recall and send information in a memory system.

Computers

Memory inside of a computer is random-access memory(RAM), contrary to sequential access memory. All of the items in a single list, or array, are stored in RAM. The computer has address decoders, similar to the way neurons work in the brain, and return items from the array that match or are similar. Each address in an array points to an individual line in the memory. That line is then returned if it is similar to other lines.

Example

Sparse distributed memory is based on pulling in patterns between different addresses.

Imagine each line as a different memory address, an example from Kanerva's book:

"Why are fire engines painted red?

Firemen's suspenders are red, too.

Two and two are four.

Four times three is twelve.

Twelve inches in a foot.

A foot is a ruler.

Queen Mary is a ruler.

Queen Mary sailed the sea.

The sea has sharks.

Sharks have fins.

The Russians conquered the Finns.

The Russians' color is red.

Fire engines are always rushin'.

So that's why they're painted red!"

As a result, all of these addresses are returned to the user, although these may not be the only addresses in that list.

Uses

"Realizing forgetting"

Decay Functions

File:Exponential Decay Function.png The exponential decay function File:Negated sigmoid function.png The negated-translated sigmoid function
x	Independent variable
a	Constant
c	Counter

At the University of Memphis, Uma Ramamurthy, Sidney K. D’Mello, and Stan Franklin created a modified version of the sparse distributed memory system that represents "realizing forgetting." It uses a decay equation to better show interference in data. The sparse distributed memory system distributes each pattern into approximately one hundredth of the locations,Template:Clarify so interference can have detrimental results.

Two possible examples of decay from this modified sparse distributed memory are presented

Exponential decay mechanism: $\!f(x)=1+e^{-ax}$

Negated-translated sigmoid decay mechanism: $f(x)=1-[{\frac {1}{1+e^{-a(x-c)}}}]$

In the exponential decay function, it approaches zero more quickly as x increases, and a is a constant(usually between 3-9) and c is a counter. For the negated-translated sigmoid function, the decay is similar to the exponential decay function when a is greater than 4.

As the graph approaches 0, it represents how the memory is being forgotten using decay mechanisms.

Genetic memory

Genetic memory uses genetic algorithm and sparse distributed memory as an artificial neural network. It has been considered for use in creating artificial life.

LIDA

LIDA uses sparse distributed memory to help model cognition in biological systems. The sparse distributed memory places space is recalling or recognizing the object that it has in relation to other objects. It was developed by Stan Franklin, the creator of the "realizing forgetting" modified sparse distributed memory system.

Implementations

• Implementation of Sparse Distributed Memory in Github: This implementation has been used for research in Fundação Getúlio Vargas headed by Ph.D. Alexandre Linhares. It was first developed to study Critical Distance and it was tested checking with Kanerva's book theoretical results.

References

43 year old Petroleum Engineer Harry from Deep River, usually spends time with hobbies and interests like renting movies, property developers in singapore new condominium and vehicle racing. Constantly enjoys going to destinations like Camino Real de Tierra Adentro.