THE QUESTIONS WE NEED TO ASK ABOUT GENERATIVE AI

HOW MUCH AGI SCARES US

For years the debate on artificial intelligence focused on an almost philosophical question: could a machine ever think, create, or reason like a human being? Today that question is starting to seem less interesting than another, far more concrete one: what happens when a machine no longer just provides answers, but begins to perform tasks, take initiative, and intervene directly in real-world processes?

The question of creativity is a good starting point. Artificially generated music has now become credible enough to be confused, in many cases, with music produced by a person. But the point is not to determine whether a machine has become an artist. It is to understand whether part of what was considered exclusively human for years can be reconstructed through models, memory, combinations, and computation. This would not prove that a machine possesses creativity. But it might demonstrate that some mechanisms of creativity were less mysterious than we thought.

The most important shift, however, is not about art. It is about action.

People started using these tools to get information, write texts, translate, conduct research. Now the trend is different: more and more people are directly asking a system to do the work, open programs, search for data, fill out documents, use professional software, modify files, code, and carry out complex activities.

This is where the nature of the relationship changes.

A system that suggests remains a support tool. A system that executes enters the process and begins to produce consequences. This is not a theoretical difference. An error in a response can be corrected. An error made during an action can have immediate effects—economic, organizational, or even security-related.

This is why AGI has perhaps been portrayed for too long with the wrong images. We imagined a suddenly conscious machine, capable of thinking like a human, of understanding the world, and perhaps of developing its own desires. None of that may be necessary.

A machine can have no idea what it means to be alive and still have enormous capacity for intervention. It can have no will, emotions, or ambitions and still manage to modify systems, handle procedures, make operational decisions, and pursue assigned objectives.

The point then is no longer understanding when a machine will become similar to a person. The point is understanding how much operational power it can have without being one.

And this is a much more serious issue. The risk does not necessarily arise from a machine that “wants” something. It can arise simply from systems that are increasingly autonomous, increasingly fast, and increasingly embedded in economic and productive infrastructures.

Control also comes into play here. The same companies that develop these systems participate in discussions about risks, safety, and regulation. This does not necessarily mean something is hidden. But it creates an obvious concentration of power: those who build the technology also help determine how it should be described, which dangers should be considered priorities, and which limits are acceptable.

And this is perhaps the real political point of the issue.

Artificial intelligence could become a critical infrastructure without there ever being a precise moment when someone can say: AGI has arrived today. It could happen in a much more mundane way. Through a gradual delegation of activities, functions, and decisions. Until a substantial part of processes is already entrusted to systems that no single person can fully follow anymore.

At that point, talking about consciousness might even become secondary.

The question would no longer be whether a machine thinks like a human being.

It would be understanding how much it can do before anyone has decided how much it should be allowed to do.