Neural Nettwork (ANN) Extension

[jrmrhrb00]

AB,

My first program (FPNQ191.000) is basically LitDev's program reworked for an "or" ann. It treats the training data as text data. Be sure to uncomment the line that writes the training data. The second program (FPNQ191.000.02) does the same thing, but I am trying to get it to work with binary data. I don't want to change the neuron structure. I just want it to work with binary data instead of text data.

LitDev, could you look at these 2 programs and give your opinion on why the second doesn't work? Maybe, it's a bug!

Thanks,

JR

[litdev]

Comments, I think mostly covered by AB, but I look at it from scratch.

1] There are 2 hidden layers in second case - no issue.
2] The data created is just appended to the text file as a continuous string of 1s and 0s - no way to know what is input or output.
000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111000101011111
3] No point in repeating the same training data 100 times, epochs in the training will handle this - all traing data provided should be independently created - for this very simple test case, there are only 4 possible cases for the OR operation (C = A or B).
4] To use binary input data, the ascii input needs to be converted to binary, see command NeuralNetwork.ConvertFile(asciiFile, binaryFile, "False").
5] To re-use a training, you should save it AFTER the training is complete, not before.

Assuming you are using most recent SmallBasicANN - I did make a few changes a while ago to generalise things a bit.

Your code modified, FPNQ191.000-3

And if you update SmallBasicANN to version 1.2.0.7 (using Extension Manager) you can use other deliminators when converting text to binary.  With a lot of training data it is quicker to append to the file as training data is created rather than create one large data string, VFWP848.000

[AbsoluteBeginner]

Since I am also new to using our ANN, please explain to me what the words "binary data" mean? 

At this point I think that "binary data" consists solely of the logical values "0" and "1".
When working with this type of data, the neural network will not look for a function that allows it to most accurately describe the pattern between input and output data.
At this point, I assume that when working with binary data, the neural network is only trying to teach each of its neurons to accurately identify only one separate combination of input data. That is, the neural network solves the problem of CLASSIFICATION. I think that this is why the output values of such a neural network will always consist only of "0" and "1".

I am very interested in studying the work of Small Basic neural networks.
But you see how slowly I work. It will probably be two or three months before I can study the neural network again. But I really want to know the answer to the question about binary data right now.
Therefore, I ask you to explain the features of the Small Basic neural network now, if anyone already knows this.

Thank you. 

[litdev]

Text and binary refer to how the data is stored in a file.  The data itself is just a set of numbers, can be integers, real or even just 0 and 1s.

If we store the data as text in the file, we just store the numbers with one byte per decimal character, reasonably efficient when the numbers are just 0 or 1, but if the numbers have loads of figures like say 3.14159265358979 (the standard number of characters for a real number in SB), then this is 17 bytes including a deliminator space character.  The number can be stored with just 4 (or maybe 8) bytes if it is stored as binary.  Text data in a file is human readable, binary data is not, but may take less space and be faster to read and write since it is already in the internal memory binary format for a number.

Google 'text and binary file storage'.

[AbsoluteBeginner]

Thank you very much. 

I thought this was not related to data storage, but to the operating mode of the neural network.

Well, while I'm making "Retro Football" I don't plan to use a neural network.
Therefore, I will watch someone else's research. 

[jrmrhrb00]

LitDev,

So, binary data should be separated with a comma or a space. Can you give an example of ASCII data in a program using the small basic neural network? I don't think there has been any programs created using ascii yet. Also, I see in your corrected program that you didn't use: NeuralNetwork.ConvertFile(trainingDataText,trainingDataBinary,"False")because it was already binary. I take it as that command did nothing.

Farher down you did use: epoch = NeuralNetwork.Train(name,trainingDataBinary,"True"),but we never create that file. So, I don't understand that. The only file that was created is trainingdatatext. We didn't create trainingdatabinary, but if the command works like it shows it uses that file for training which doesn't make any sense to me.

JR





I did update to 1.2.0.7.

[litdev]

Hi, maybe some misunderstandings here.

Ascii or binary refer to the file type that the training input data is stored, it is nothing to do with the fact that the data is 1s and 0s.

The data is initially stored in ascii (txt, non binary) format, in this case the data has to be delimiated by some character, " " or "," etc.  Before the most recent update today (1.2.0.7), each value of input training data had to be on a new line.  Open and look at the contents of orANNInput.txt file.

The command NeuralNetwork.ConvertFile(trainingDataText,trainingDataBinary,"False") converts the contents of trainingDataText.txt into a new binary file, creating the file defined by trainingDataBinary.  Open orANNInput.bin and look at how the same data is stored as a binary file.

The last argument ("True") of training command epoch = NeuralNetwork.Train(name,trainingDataBinary,"True") training command says that the file format of the training data is binary.

The other place where binary is mentioned is that the output is binary, option NeuralNetwork.BinaryOutput(name,0,"true").  Internally when using the ANN (NeuralNetwork.Use), the output will be calculated as a value between 0 and 1.  This option will just convert any result < 0.5 as 0, and a value >= 0.5 as 1.

We don't have to convert to binary input data - the following also works, training by reading the ascii input data, so the option to convert to binary file type is probably not that important really.

'*** NeuralNetwork.ConvertFile(trainingDataText,trainingDataBinary,"False") 'Commented out
textwindow.writeline("Training started")
epoch = NeuralNetwork.Train(name,trainingDataText,"False")

The fact that your input data is just 0s and 1s is not relevant to any of the discussions above.

Hope this is clearer.

[jrmrhrb00]

LitDev,

With the update to 1.2.0.7 and using your new program VFWP848.000 I show 4 lines of data: 0 0 0
1 0 1
0 1 1
1 1 1. which is what it should be. The .bin file is weird. It shows what looks like 7 8's followed by question marks. So, why would that be?

ð? ð? ð? ð? ð? ð? ð? The hex character to create that is 3f. Why it makes the file I don't know.

JR

[litdev]

JR,  The binary storage is the 8 byte representation of each double precision number - google this, its just a set of 8 bytes for each number.  When you open the file in a text editor, each of these bytes is converted to a character using ascii character codes. HEX 0 does not have an ascii character so mostly you don't see anything and what you do see is nothing to do with the values of the data.  If you have it you can open the file in a HEX editor and see the individual byte values, rather than them being wrongly converted to ascii characters. Visual Studio below, but I think there is a HEX plugin for Notepad++ for example.

   

[jrmrhrb00]

LitDev.

I updated the thread. The goofy characters are due to 3f hex value. So, there is really nothing in the file. The interesting thing is you do needt he file. I commented out: NeuralNetwork.ConvertFile(trainingDataText,trainingDataBinary,"False") and deleted the .bin file. When I ran the program it said: epoch=Error: training set file doesn't exist.

Thanks for all the help on this. I know it's confusing, but I think it can be a great extension.

JR