On the Nature of Generative AI Intelligence

It is amazing how today's Artificial Intelligence systems are able to do things we could not even imagine a few years ago: they answer our questions in our own language, they translate from one language to another much better than in the past, they recognize images and generate new ones, they are able to reproduce a person's voice.

The results achieved in the last couple of years are simply astounding.

Generative Artificial Intelligence

Generative AI, the type of Artificial Intelligence that has recently taken off and that everyone is talking about, is based on a simple and therefore all the more surprising principle: it is the principle of “what comes next?”

You know that function on your smartphone that suggests a word when you type a message? There it is, that's the basic principle of generative AI, of course with all the appropriate evolutions.

Even the more sophisticated LLMs (Large Language Models), the systems that power AI tools like ChatGPT, Copilot and Gemini, are based on the principle of predicting words that follow those in a given context. They base their predictions on the experience gained from analyzing millions of texts used during training. And the more context you give them in your questions, the more accurate their answer is, just like humans.

Of course, the mechanism is not limited to text. You can apply it to other media such as images, video, audio and so on. Instead of predicting the next word, they predict the next “token.”

The fact that you can engage in discourse and generate content based on this principle seems to me truly amazing. And the results are undoubtedly practical as well. We are seeing this and I’m sure we will see other interesting applications in the near future.

Is prediction reasoning?

This approach to Artificial Intelligence (because it is not the only approach) is getting people excited, so much so that many are asking how far such technology can go, whether human intelligence is threatened, whether robots will replace us, what the legal and moral implications are, and so on.

It must be said that since the 1950s, with each wave of enthusiasm for Artificial Intelligence (there have been others in the past), the usual diatribes about whether or not machines can think or not, whether or not they are really intelligent, whether or not they have consciousness, etc. have arisen.

For example, some time ago I came across this thread discussing why the next-word prediction principle leads to real understanding. The discussion was based on a video of an interview with Ilya Sutskever, co-founder of OpenAI, who used the metaphor of the detective in a mystery novel who, after collecting all the clues, “predicts” who the culprit is. In short, he seems to be arguing that predicting something given a context is basically nothing more than reasoning, and therefore understanding. Needless to say, this left me a bit puzzled, especially since the comparison comes from someone who knows how generative AI works.

Let us return for a moment to the behavior of an LLM that I mentioned earlier. As it stands, an LLM is a system that recognizes a pattern based on the data analyzed during its training. Basically, when we chat with it, the system decides how to continue its sentences based on what it has seen previously with a verbal context similar to the current one.

If you think about it, it is not so different from what we do when we speak: we subconsciously decide the next word to say based on what we have said before. We also do this when we understand speech: we decide the meaning of a word we have not heard well based on the context of the words we have already heard. We can also predict the next word someone will say based on what they have said so far. It is an instinctive, automatic mechanism.

A reflex… to reflect on

But if this mechanism is instinctive, if it is automatic, it means that we have not thought about it. It is not the product of any particular reasoning, that is to say, of our intelligence. It is just a reflex.

Yes, that is exactly what it is. It is a reflex acquired through thousands and thousands of hours of practice.

It is a bit like me saying to you, “Peter Piper picked a peck...,” and now you go on. Without even thinking about it you answer “...of pickled peppers.” It's a reflex.

We have so many of them. Nature (or evolution) has provided us with several for our safety: if something suddenly approaches our eye, we immediately close our eyelids; if we lose our balance, we instinctively extend our arms. We don't think about the fact that our center of gravity has shifted and we need to bring it back to a point where we won’t fall.

In addition to innate reflexes, we add others that we can call skills, such as speaking, writing, singing, and so on. These skills are the result of continuous practice, training. Just like athletes or musicians: when you start to play tennis, you stand there and think about how to hold the racket, how to react to a shot, etc.; when you learn to play the piano you stand there and see how to set up the left and right hands, which key to press to play a certain note, etc. Then you don't do it anymore. It has become a mechanism.

At this point I wonder: is acquiring these mechanisms and using them intelligence?

Who is intelligent?

Okay, okay. This is a tired story: what is intelligence? Is who seems intelligent really intelligent? I am reminded of the phrase Forrest Gump used to repeat : “Stupid is as stupid does”. There are several metaphors and discussions about considering those who act intelligent as intelligent: from the Turing test to the Chinese room.

But beyond that. Would you feel comfortable saying that it takes intelligence to acquire unconscious behavior? Are domesticated animals intelligent?

Building these systems capable of acquiring skills is certainly an important step on the journey to exploring intellectual activities. But these are skills like memorizing a large amount of data or doing calculations quickly. Nothing more. By the way, dear old calculators are great at doing even complex calculations quickly, but no one has ever considered them intelligent. How come?