1. Introduction to the Problem
Where did neural networks come from, and why are we unsatisfied with them?
The development of artificial neural networks began with the work of Turing, McCulloch, Pitts, and
Hebb. Based on their ideas, in 1958, Frank Rosenblatt created the first artificial neural network,
«Perceptron,» capable of recognizing and classifying different objects based on recognition after appropriate training. Unfortunately, the very concept of the perceptron was fraught with a critical flaw, based on then-prevailing Dale’s biological doctrine: «…A neuron uses one and only one neurotransmitter for all synapses.» This doctrine was transferred to all artificial neural networks through a rule: «…One artificial synapse uses one and only one synaptic weight.» This rule may be called the Rosenblatt Doctrine.
In the 1970s, Dale’s doctrine was rejected by biological science. Unfortunately, Rosenblatt’s doctrine remains unchanged for all neural networks (recurrent, resonant, deep, convolutional, LSTM, generative, forward, and backward error propagation networks). This doctrine makes it possible to train networks using an iterative approach known as the gradient descent method, which requires enormous computation. And it is precisely this doctrine that is «to blame» for the inability to construct an adequate working theory of neural networks. Also, these networks are characterized by opacity and incomprehensibility, relatively low training speed, difficulty in completing training, and many other innate problems. For more information on the issues of classical neural networks, see Appendix 1.
Therefore, the development of such networks is mainly by trial and error. This leads to complexity and low reliability, the need for costly equipment, conducting complex power-hungry calculations, and expensive manual labor to provide training.
The critical «Rosenblatt error» was discovered by researchers (TRIZ specialists) of the deep tech company Progress, Inc. They also found a solution to eliminate this error. Thus, it became possible to create a fundamentally new type of neural network called PANN (Progress Artificial Neural Network). PANN networks and their operations are transparent, predictable, and thousands of times less costly, providing a better solution to many intelligent tasks. Eighteen patents in many countries worldwide protect PANN’s designs and operations. Several new software versions have already been created and tested based on these concepts.
2. Scientific and technical foundations of the PANN network
In this chapter, we will describe the main design features and the theoretical basics of the PANN network.
PANN differs from classical neural networks in that it has a unique design for the main element: the so-called formal neuron. A new formal neuron allows for a different way of training. As a result:
1. The network operation has become completely transparent. Establishing a simple and straightforward theory that predicts the results of actions has become possible.
2. PANN can be implemented on low-cost hardware. Its training and operation costs are much lower than those of classical neural networks.
3. PANN trains many times faster than classical neural networks.
4. PANN can be trained to additional (new) data anytime.
5. PANN does not have the harmful effect of «overfitting.»
2.1. A NEW DESIGN OF THE FORMAL NEURON
Classical neural networks are built of typical «bricks» – formal neurons of simple design, described by McCulloch and Pitts and implemented by Rosenblatt. The main problem with neural networks is the poor design of this formal neuron.
A formal Rosenblatt neuron has one synaptic weight. The PANN’s unique feature is a formal Progress neuron with two or more synaptic weights at each synapse.
Fig. 1. Comparison of formal neurons
In the Progress neuron, as in the Rosenblatt neuron, input signals travel to the adder through a single synaptic weight. However, in the Progress neuron, the distributor selects the weight based on the input signal size.
