Dynamic Synapse: Concept and Applications

In this seminar, I'll present the concept of Dynamic Synapse and review some of the applications using neural networks that incorporate dynamic synapses. In a conventional neural network, a synapse is represented as a number, called synaptic weight, which specifies the strength of the connection between neurons. A neural network can be trained to perform a desired task by changing the synaptic weights according to some learning rules. By representing the synapse as a single number, the conventional neural network is faced with two fundamental limitations: First, although a neuron can be connected to a large number (thousands) of other neurons, it can send only one signal to all these neurons. Second, only the synaptic weight can be tuned during learning which, amounts to merely changing the magnitude of the output signal of a neuron.

Neurons in the brain interact with each other by transmitting sequences of electrical impulses via synapses. Although a number of dynamic processes have been known to exist in the synapse, their role in neural information processing had been unclear. The concept of dynamic synapse was developed as an attempt to explain the function of these dynamic processes: With dynamic processes, a synapse is capable of transforming a sequence of electrical impulses into another sequence of impulses. That is, a synapse can perform pattern transformation function. Furthermore, variations across the many synapses of a single neuron lead to different transformation of the impulse sequence. As a result, dynamic synapses allow a neuron to transmit multiple output signals. We have developed a learning algorithm, which trains each dynamic synapse to perform a proper transformation function such that the neural network can achieve the desired task. To exemplify these concepts, I will review some applications, including word recognition, speaker identification, robot sensor fusion, and biosonar recognition.