Training program for SB-Neuron.

[AbsoluteBeginner]

(translated by Google translator)

Hi all. 

So, we have found out that the quality of the responses that a neuron gives in response to input data is determined by the set of weights of the neuron's inputs and the shape of the activation function.

Almost always we hear that during training a set of weights for the neuron inputs is configured.
We know that before training we can choose the type of activation function for a neuron.

But our forum will not be worthy of the proud name "Small Basic" if we do not try to create such a training program that will SIMULTANEOUSLY configure BOTH the input weights of the SB-Neuron and the shape of its activation function. 

Let's do it, gentlemen!..

[AbsoluteBeginner]

 

The great Konosuke Matsushita once said: "It is foolish to ask a man what he wants. Show him the thing so that he will know what to want."

Practice shows that this is correct.

So, to help us understand what we need to do to create a training program for SB-Neurons, we need a demo program that visualizes the training data.

We will see with our own eyes WHAT the SB-Neural Network must be able to do in order to successfully recognize images of "crosses" and "zeros" on a matrix measuring 11 by 11 pixels.

And let's start, perhaps, with information about the image of the "feature" measuring 3 by 3 pixels.
(the “feature” are discussed in the article on convolutional neural networks at the link provided by LitDev.)

https://e2eml.school/how_convolutional_n..._work.html

[AbsoluteBeginner]

(translated by Google translator)

Hi all.

I have posted in my OneDrive a screenshot "DemoTrainingProgram screenshot.jpg" of the current version of the demo training program for SB-Neurons.
https://1drv.ms/f/s!AnoSlTzMqlL6jNx6H4Qi...w?e=8D73FE

In this screenshot you can see two graphs.
The left graph shows the real training data for a SB-Neuron that has 4 logic inputs. The inputs of this neuron will be fed with "0" or "1", which will correspond to the color of a pixel on the image "feature" matrix, measuring 2 by 2 pixels.

Given this input, our trained SB-Neuron will determine the TYPE of the image "feature".

As you can see, the real training data is quite scattered across the graph.
This is clearly visible on the right graph of the screenshot, where the real data is already shown as red dots.
When real data is so widely scattered, it is very difficult for a classical neuron to learn to give accurate answers.

On the same right graph, you will see a blue line that depicts the ideal shape of the neuron activation function for such a task.
But to work successfully with such an activation function, our SB-Neuron training program will have to select new weights for the neuron inputs.

Once the demo program is created, we will be able to see on the right graph how the red dots of the neuron's responses will gradually approach the blue target line.
I hope the experiment will be a success. And for the first time in the history of Humanity, we will perform a SIMULTANEOUS setup of both the activation functions and the weights at the inputs of our proprietary SB-Neuron. 

Right?.. 

[AbsoluteBeginner]

(translated by Google translator)

Hello, friends. 

I can tell you that I was unable to get sufficiently accurate answers from the SB-Neuron using weight changes on its inputs.
For the second time in a row, I can declare the victory of the "Create an individual activation function" method over the "Change neuron weights" method.
I think that weight changes will only be an auxiliary tool when creating SB-Neurons in the future.

So, in the task "Recognize the image of a "cross" or "zero" on an 11 by 11 pixel matrix" we are ready to begin the next stage of work.
Now we can begin creating a SB-Neural Network that will be able to recognize one fixed version of the "cross" image and one fixed version of the "zero" image in an arbitrary place in the matrix.
We will create a neural network that recognizes an image by "features" later.

[AbsoluteBeginner]

(translated by Google translator)

Hello, friends.

This is what I think about all the time.
From the very beginning of my study of the SmallBasicANN extension, I really didn't like the fact that the structure of the neural network had to be selected experimentally.
I wanted to be able to precisely DESIGN the neural network, like electrical circuits are designed.
I wanted to be able to first train a neuron the way I needed, and then place it in the neural network where I needed it according to my plan.

Now that we have very high-quality SB-Neurons that do not require preliminary data preparation for their operation, I feel very free.
It's as if my hands have been untied.
I have the ability to PLAN how the process will proceed in the SB-Neural Network I created.
In addition, I am confident in the high accuracy of the data that each of my SB-Neurons produces.

So I will solve the problem of "Recognize the image of a "cross" or "zero" on a matrix of 11 by 11 pixels" NOT as a passive observer who simply tests random structures of a neural network and waits for some structure to be able to work. Now I will ASSEMBLE a SB-Neural Network from pre-trained SB-Neurons, like a LEGO user assembles his model from parts of a strictly defined shape.

And you, what do you think?.. 

[litdev]

I'm not fully sure what your approach is, and we have many discussions on traditional ANNs and how I believe they work.

From what I understand it may a bit different from traditional AI (which generallly uses an ANN to that is trained to a specific task, that can equally be trained for any number of other tasks - the ANN is generic and the training is specific to a task).  I think what you are proposing is more like what used to be called 'expert system', where the program encodes understanding of knowledge specific to the task.  

A subtle difference I know - for an expert system the programmer encodes their 'expert' understanding of the problem, and in AI, the program works out its own 'understanding' through extensive training.

Example of expert system might be a set of pre-programmed questions that a Doctor might ask to diagnose a patient's problem and suggest a remedy - this expert system could not be retrained to diagnose a household plumbing issue. 

Example of AI is to look at a brain scan and detect early onset Altzeimer's desease - the same ANN could be trained to detect dogs or boats in an image.

So, the task of detecting any knind of X or O drawn in any place of any size on 11x11 grid that is recognisable to a human as X or O is a reasonable challenge and could be approached with either approach I think.

[AbsoluteBeginner]

LitDev,

I hope that in this message I will be able to describe more precisely the DIFFERENCE between what I want and the neural networks you are talking about.

It seems to me that this difference is that "classical" neural networks are assembled from untrained neurons and the entire network is trained as a whole. For this reason, these neural networks have serious shortcomings.

But I am interested in assembling a neural network, all of whose neurons are ALREADY TRAINED.
That is, training of an already assembled network is not carried out.

And training of individual neurons to solve their simple problems is exactly the style of our Small Basic: simplicity and excellent capabilities. 

In general, our SB-Neural Network is a CONSTRUCTION consisting of pre-trained SB-Neurons.
The fact that SB-Neurons are trained in advance allows you to consciously construct a neural network from them that has the necessary capabilities.

(I hope that now I have described everything more clearly )

[litdev]

AB,

Yes I understand what you are trying to achieve and the 11x11 grid X,O is a very good test for it, good luck

[AbsoluteBeginner]

(translated by Google translator)

Hello, friends. 
For three days now I have been entertaining myself with researching "features" for use in a neural network.
I deliberately did not read other people's research so that I could enjoy my own discoveries in this world.

And today I gave up.
I couldn't figure out how to explain to the neural network during training that it should find a STRAIGHT line.
I could not formulate the "feature" of the Straight Line.
And now, I'm going to start reading other people's research on this topic.

What about you? What's new with you? 

[AbsoluteBeginner]

(translated by Google translator)

Hello, friends. 

My entertainment with the creation of SB-Neurons and SB-Neural Networks gave me another pleasure.
I learned that the methods that other people use to train classical neurons to recognize a straight line on a matrix of pixels are more complicated than my method.
I think this is because I will be using SB-Neurons, not classical neurons.
As you know, SB-Neurons have an advantage because they use the principle of changing the activation function, instead of the principle of changing the weights on the neuron's inputs.

Now that I understand everything I need to create a SB-Neural Network, I have the opportunity to write the SB code of a new demo program.
This program will demonstrate how my SB-Neural Network will recognize a "cross" on a matrix of 9 by 9 pixels.
The "cross" can have any size, rotation angle, and position on the matrix.

The neural network will not look for the "cross" itself, but for a set of "features" that characterize any normal "cross".
This is the "end of the line" feature;
the "presence of two pairs of "ends of the line" of opposite directions on the matrix" feature;
the "intersection of two lines" feature;
and the "straight line" feature.

If all of the listed "features" are detected, my SB-Neural Network will say that a "cross" is depicted on the matrix.

This is the kind of entertainment I have at the moment.

(It's a shame that no one else talks about their interesting SB-projects )