Yesterday, 09:54 AM
It does seem like a mind, but...
What it actually is is a very big trained neural net, that takes input text (for a LLM) and is trained to output what it has been trained is the most likely next word, and it does this word by word!
GPT-3 has 175 billion parameters in its neural net, and was trained on the entire www at the time - that is every book, technical paper, bit of softawre code, blog, image, everything with no regard for privacy, accuracy etc. This takes enormous computing power and is hideously expensive in hardware and power - each GPU costs $50,000 and there are hundres of thousands of them doing this training. GPT-4 and successors are orders of magnitude bigger.
The actual neural net code is very simple node by node, much the same as the ANN in SB, but the amount of data and training required is astonomical. The real value is the data required and that is why Google, Apple, Meta et al want everything they can about everything and especially us individually.
A human brain neuron is also relatively simple individually, but with billions of them and their connections consciousness and intelligence emerges. I don't think we are anywhere near that with generative AI, but that is philosophical. Remember it is just an enormous neural net trained to output what it thinks is most likely to come next. It will always sound totally authorative and will be right a lot of the time - when it is pushed into areas it hasn't been trained on it will make stuff up very convincingly.
What it actually is is a very big trained neural net, that takes input text (for a LLM) and is trained to output what it has been trained is the most likely next word, and it does this word by word!
GPT-3 has 175 billion parameters in its neural net, and was trained on the entire www at the time - that is every book, technical paper, bit of softawre code, blog, image, everything with no regard for privacy, accuracy etc. This takes enormous computing power and is hideously expensive in hardware and power - each GPU costs $50,000 and there are hundres of thousands of them doing this training. GPT-4 and successors are orders of magnitude bigger.
The actual neural net code is very simple node by node, much the same as the ANN in SB, but the amount of data and training required is astonomical. The real value is the data required and that is why Google, Apple, Meta et al want everything they can about everything and especially us individually.
A human brain neuron is also relatively simple individually, but with billions of them and their connections consciousness and intelligence emerges. I don't think we are anywhere near that with generative AI, but that is philosophical. Remember it is just an enormous neural net trained to output what it thinks is most likely to come next. It will always sound totally authorative and will be right a lot of the time - when it is pushed into areas it hasn't been trained on it will make stuff up very convincingly.
![[-]](https://litdev.uk/mybb/images/collapse.png)