Neural Nettwork (ANN) Extension

[AbsoluteBeginner]

(translated by Google translator)

Hi all. 

Dear LitDev,
may I know your opinion about the extension for working with neural networks?
Is it worth spending time studying it?
And could this extension be interesting for use in the "AI Snake" Championship?

Thank you.

[litdev]

I can't see any existing links, code or any version of dll for it (I assume the one by EzyMCP).  If some-one has it then we could have a look.

It might be interesting to play with it in a general sense, but probably not directly for your AI program.  You would need a large training set of data for it - (like thousands of human played games).  Think of a neural net as a complex curve fitting type program, that finds the best fit by tuning coefficients to match a large set of training data.

[AbsoluteBeginner]

I can't see any existing links, code or any version of dll for it (I assume the one by EzyMCP).  If some-one has it then we could have a look...

I asked about the “SmallBasicANN” extension, which I saw in the list of “Extension Manager” of the SB-Prime editor. I saw this when I installed the "SB Forms" extension to look at it. 

[litdev]

Yes, I forgot about that one (it was originally by EzyMPC so I must have had the code for it at some point) - I haven't really used it and can't find any samples for it so it may be fun to look into it, but my comments about ANNs (Artificial Neural Nets) in general and to your AI program will still apply.

[AbsoluteBeginner]

I understand.

But the gambling devil-inventor who lives inside me is already whispering in my ear: “We need to hold a competition between the neural network Mind and the Mind that works according to other principles.”. 

Thank you very much, LitDev.
I wish everyone a good holiday.

[litdev]

A couple of notes regarding ANN extension, if anyone is interested in maybe using this extension for something more serious.

1] The inputs and outputs both need to be normalised to be between 0 and 1.
2] Corrected a couple bugs (using text training data, 2 issues) and added option to set non-binary output - need to update using extension manager.

Here is my test code (a little tougher than XOr training, but not so much really), averaging 2 integers between 1 and 100, ZWWL934.000.

Results running 10 tests using ANN after traning with 10000 data points, the ANN results are in brackets.

Training started
epoch=1
Trained=1
BinaryOutput=False
Epoch=100000
LearningRate=0.7
Momentum=0.3
SigmoidResponse=1
ErrorRequired=0.0001
Mean(30,87) = 58.5 (59.467565069791200)
Mean(88,94) = 91 (83.865182777482700)
Mean(84,86) = 85 (81.174412270905500)
Mean(76,23) = 49.5 (48.755414992063700)
Mean(33,43) = 38 (35.396646012457900)
Mean(7,43) = 25 (23.578500109811300)
Mean(52,73) = 62.5 (63.828703608754200)
Mean(85,69) = 77 (76.402047305511200)
Mean(88,82) = 85 (81.147569198575100)
Mean(85,38) = 61.5 (62.70809712805100)
Press any key to continue...

[AbsoluteBeginner]

Thank you. 

Personally, I'm interested in being able to use neural networks when creating programs in Small BASIC.
It's fashionable now.
Our Small Basic is good enough that it doesn't lag behind others. 

[litdev]

AB,
Great - go for it.  I just fixed another small bug where Sigmoid cannot be set correctly.

[AbsoluteBeginner]

(translated by Google translator)

Hi all. 
Yesterday I conducted tests on my neural network, which should learn to convert the Color value from the RGB format to the HSL format.
The first thing I want to say is my joy that I have the opportunity to see reports while training a neural network.
For now, I'm just displaying the entire report array in the title of my text window.
I noticed that if I set the value of the "NeuralNetwork.ProgressReportFrequency" parameter to 1 more than the number of lines in the training data file, then the information in the title of the Text Window is visible well.

If anyone knows any other life hacks for using the Small Basic neural network, please tell us here. 
Now I’ll transfer the files from my tablet to my powerful desktop PC and try to fully train the neural network.

( by the way, who knows whether the result of training a neural network depends on the SEQUENCE OF LOCATION of lines in the training data file? In my file now the training data is arranged in ascending order of input values. Maybe it would be better if this data was located randomly in the file? )

Thank you.

[litdev]

AB,

My understanding is that training is usually done with data in a random order, but good for you to try different approaches.

Also, it is easier to trainl HSL->RGB rather than RGB->HSL.  This is because Hue 0 is the same as 360 and 1 is very close to 359 etc.  I had better results training RGB to Cos(H),Sin(H),S,L (all of cource normalised appropriately between 0 and 1).