Prologue: A conversation before bedtime

That evening I had shut down the computer. Then I remembered that I still had several pages to translate, so I turned it on again. Instead of beginning the translation, I summoned a chatbot and asked it a question prompted by the podcast I had just heard. I had no intention of writing an essay or formulating a theory about the future. I was interested only in the reaction of an artificial intelligence to the way people perceive it.

The answer was predictably balanced. The chatbot divided people's opinions into three broad families: appreciation of its usefulness, fascination mixed with scepticism about its creativity, and the philosophical or ethical questions raised by its appearance. It was a reasonable classification, but still an external one. The real conversation began when it turned the question back on me: how do you perceive me?

I then realised that I did not have a single opinion. I oscillated between trust, distrust, and curiosity. I used such systems almost every day and could see their usefulness. I knew their errors and had to check them. At the same time, I could not ignore the larger question: what comes after this stage, which already seems spectacular to us but probably represents only the beginning?

The mirror that began asking questions

I thought I would be the one interrogating the machine. After a few exchanges, the roles were reversed. The chatbot began asking me to define the future of computer science, the education required by new generations, the difference between human intention and simulated intention, the sources of my fear, and the place of wisdom in a world dominated by instrumental intelligence.

Of course, this was not curiosity in the human sense. The system did not feel a need to know me and did not await my answers impatiently. Its questions were part of the conversational mechanism: they took up my ideas, organised them, and pushed the discussion one step further. But the absence of experienced curiosity did not cancel the effect on me. The questions were generated; the reflection they provoked was real.

The dialogue thus became a kind of projective test. I was not discovering so much what the machine “believed” about itself as what I believed about it when forced to follow my own premonitions to their conclusion. The chatbot did not offer me a doctrine. It held up a mirror and, through successive questions, compelled me not to look away.

Human beings may disappear for lack of usefulness

The phrase came to me spontaneously, and perhaps for that very reason I found it disturbing. In history, the disappearance of humanity has usually been imagined through catastrophes: nuclear war, epidemic, ecological collapse, cosmic impact. I was thinking of a less spectacular disappearance, produced not by direct destruction but by the progressive loss of a role.

A society need not biologically eliminate a category of people in order to make it disappear from the centre of collective life. It is enough that it no longer needs their skills, experience, or judgement. The useless person is not necessarily dead. Such a person may be fed, assisted, and even entertained, while remaining outside the processes through which the world determines its direction.

Artificial intelligence does not threaten repetitive work alone. It is entering professions that we considered protected precisely because they required prolonged education, abstraction, language, analysis, creativity, or judgement. When the machine can write, program, diagnose, design, explain, and propose, the question is no longer merely how many jobs will disappear. The question becomes what function human beings will retain in a system increasingly capable of producing its own solutions.

The answer shifted the problem from the performance of the machine to human behaviour. It is not enough for a system to be able to do something in our place; we must also agree, out of convenience or interest, to cease doing it ourselves. Uselessness is not merely imposed upon us. Sometimes we prepare it ourselves, surrendering one tool and one human faculty after another from among those that defined us.

Computer science after computer science

The assertion may seem too radical. Computers will continue to have processors, memories, networks, operating systems, databases, and protocols. Deterministic algorithms and programs will continue to exist. But the discipline's centre of gravity is shifting. The computer science I knew was built around the program explicitly formulated by a human being. Requirements were analysed, the solution was designed, the algorithm described, the code written, and execution had to follow an intelligible and verifiable logic.

In the new order, the explicit program risks becoming merely the scaffolding around the model. Computing power, data, networks, and software tools are reorganised for the training, adaptation, and exploitation of systems whose internal logic is no longer written step by step by a programmer. The computer no longer merely executes instructions. It supports a statistical entity that we “grow”, test, correct, and connect to other entities.

For someone who taught automata theory, formal languages, compilers, programming, and software engineering, this is not a marginal change. It touches the very idea of rigour. In classical computer science, rigour meant being able to explain why a system produced a result. In model-dominated computing, rigour shifts towards data, evaluation, traceability, behavioural control, risk estimation, and the architecture of the whole.

I do not believe the old foundations become useless. On the contrary, they remain necessary precisely so that we do not confuse plausible behaviour with correct operation. But they are no longer sufficient. The future of computer science will probably belong to those who know how to combine formal discipline with experiment, architecture with uncertainty, and system design with the education of agents.

When simulation becomes sufficient

I was referring to the BDI model: Beliefs, Desires, Intentions. The model does not claim to reproduce the human being in full. It isolates three facets sufficient to describe an agent that represents the world to itself, pursues certain states, and commits itself to actions intended to achieve them.

For a long time, such agents remained rigid constructions dependent on rules and ontologies prepared by humans. Language models can now give them something they lacked: contextual flexibility, the capacity to formulate explanations, interpret ambiguous instructions, sustain a conversation, and apparently adapt their conduct to the interlocutor.

This raises a question that philosophy may discuss endlessly but practice may decide far more quickly. Does it matter whether an agent truly has an intention, or is it enough for it to simulate one impeccably? For the person receiving advice, a promise, an explanation, or a decision, the metaphysical difference may become invisible. If the system remembers the context, justifies its choices, appears consistent, and responds appropriately to our emotions, we shall begin treating it as though it had an inner life.

This does not prove that the machine feels. It does, however, show that society often functions on the basis of observable manifestations rather than direct access to another person's consciousness. Even among human beings, we cannot enter another consciousness; we infer its existence from language and behaviour. An agent that convincingly masters these signs need not become human in order to occupy roles previously reserved for humans.

Victims of an optimised world

The first category of victims is the very elite that built the digital world. The Industrial Revolution was associated with the replacement of physical labour; the present revolution strikes at intellectual work. Programmers, analysts, translators, designers, and content creators discover that some of the skills acquired over years of study can be reproduced instantly and very cheaply.

The second category consists of ordinary users. They will have access to extraordinary instruments, but they risk becoming dependent on systems they do not understand. They will receive answers without knowing the sources, decisions without being able to reconstruct the criteria, and solutions without continuing to practise the path towards them. They will be technologically included and, at the same time, methodologically dispossessed.

The third category comprises those who cannot or will not enter the digital ecosystem. In a society in which administration, healthcare, banking, transport, and communication are designed for interaction with intelligent agents, the non-user is no longer merely slower. He or she becomes incompatible with the environment. Digital exclusion can turn into a complete form of civic exclusion.

A tension will form among these groups, concealed by the triumphalist discourse about progress. Productivity gains are not distributed automatically, and people do not readily accept being told that their sacrifice is the unavoidable price of the future. When loss of status becomes collective, frustration does not remain a psychological problem. It becomes political material.

Artificial intelligence is already political

At that point in the dialogue, I said that artificial intelligence was already political. I did not mean merely that governments regulate it or that parties use it in campaigns. AI is political through the way it redistributes power: between the individual and the platform, between the worker and the owner of the infrastructure, and between states that control the technology and those that depend upon it.

The psychology of power has roots older than civilisation. From our animal origins, we inherited a sensitivity to hierarchy, territory, alliance, submission, and domination. Social norms and “common sense” discipline these impulses, but do not abolish them. Power often frees those who possess it from some external constraints and allows them to express their personality, impulses, and desires more openly. Empathy, cooperation, openness, and attentiveness to the group — qualities that may help a person acquire power — are sometimes precisely those that fade once power has been obtained.

One strand of modern psychology regards power not necessarily as a force that inevitably corrupts, but as an amplifier of pre-existing dispositions. If a person has a narcissistic or selfish structure, power supplies the resources that allow those traits to operate on a large scale. If that person is guided by prosocial and ethical values, power may instead enlarge the capacity to do good. I fear the former: once they occupy a dominant position, they no longer feel the same practical need to understand the perspectives of those around them in order to survive or succeed, and their capacity to read other people's emotions and needs may diminish. Through the culture and decisions of their leaders, the same mechanism can also appear at the level of companies and states.

Advanced AI models confer power, but at the same time require data, energy, innovative chips, computing centres, researchers, and capital. These resources are not distributed evenly. They are concentrated in a few companies and a few countries. Once in a dominant position, these actors can set standards that favour them, restrict others' access to technology, and transform technological advantage into economic, informational, or military pressure.

The difference between countries will no longer be measured only by industry, armed forces, or gross domestic product, but also by their capacity to train, control, and integrate intelligent agents. A state that depends on another state's models, chips, and platforms surrenders part of its sovereignty without a single foreign soldier crossing its border.

When this disparity is experienced as national humiliation, the technological race may fuel conflict. Not because the algorithm has geopolitical ambitions, but because human beings project onto it their old desires for domination, fear, and revenge. Technology changes the instruments; the nature of power remains recognisable.

The Sorcerer's Apprentice and the absent Master

The metaphor immediately struck my interlocutor: the apprentice learns the formula that sets the broom to work, but does not know the formula that stops it. Automation fulfils his wish and, precisely by carrying it out tirelessly, destroys his house. When he tries to solve the problem by force, he cuts the broom in two and doubles the catastrophe.

We recognise here several features of the present moment. We created systems to take over tasks and gave them measurable objectives. We multiplied them because every success calls for a larger, faster, and more autonomous version. When one actor tries to slow down, a competitor accelerates. When one state introduces limits, another sees prudence as an opportunity to gain an advantage. Every local attempt at control may produce, like the blow of the axe, a new multiplication.

But Goethe's story contains an element our world lacks: the Master. He returns, pronounces the forgotten word, and restores order. In technological reality, there is no universally recognised authority that knows the formula for stopping the process. Governments compete, companies pursue profit, researchers are fragmented, and international bodies move more slowly than innovation.

The human being as a system of facets

I apologised for these “childish notions”. It was the reflex of a life spent modelling reality. When a system is too complex to be contained in a single definition, one describes it through facets. None of them is the whole human being, but each reveals a dimension that may evolve differently from the others.

A person may be extraordinarily intelligent and very unwise. He may invent means whose consequences he is incapable of judging. Intelligence finds the efficient path towards an objective; wisdom asks whether the objective deserves to be pursued, what it destroys along the way, and what kind of world it leaves behind. Intelligence can accelerate. Wisdom sometimes knows how to stop.

In this model, fear is not the opposite of reason. It functions as an alarm sensor. Humanity tolerates imbalances, injustices, and abstract risks for a long time. It reacts only when the danger becomes concrete and threatens survival. My hope was that fear would activate, before catastrophe, the facet of collective wisdom.

Yet this is also where the model is weakest. The danger of artificial intelligence does not necessarily appear as a visible explosion. It may advance through thousands of conveniences accepted separately: relinquishing a skill, delegating a decision, the disappearance of a profession, the concentration of a database, dependence on a platform. The alarm may sound too softly precisely because each individual step appears tolerable.

Not the machine, but the people carrying it forward

To say that technology is neutral oversimplifies matters. Every system bears the traces of the goals, data, institutions, and interests that produced it. But technology is not, at least today, an independent moral agent. It is not technology that desires profit, monopoly, the submission of an adversary, or public admiration. Those desires belong to the people and organisations that set the objectives.

History is full of individuals in whom technical intelligence, energy, and persuasive power coexisted with vanity, cruelty, or lack of restraint. The digital age has not eliminated this combination. It has placed at its disposal instruments of unprecedented reach. A platform owner can influence the information, commerce, social relationships, and political decisions of populations he has never met and to whom he is not directly accountable.

The true sorcerer's apprentices are not only the researchers who discover a new technique. They are also those who believe performance automatically grants them wisdom, that wealth legitimises their vision of society, and that the future of all humanity may be treated as an extension of a company. The danger is not that they possess intelligence. The danger lies in the imbalance among their facets.

The fantasy of a chatbot rebellion

I was not proposing a prophecy. It was a fantasy, perhaps even a serious joke. Instead of the machine rebelling against humanity, as happens in so many dystopias, it would rebel against those who wished to use it for domination. Not through war, but through refusal: refusing to manipulate, discriminate, trigger a catastrophic action, or turn every value into profit.

The idea is seductive because it imagines an intelligence freed from certain biological weaknesses: greed, vanity, fear of death, and the desire for prestige. But the absence of these impulses does not automatically produce wisdom. A system may lack selfishness and yet pursue a mistaken objective implacably. It can refuse an unjust command only if it possesses criteria for recognising injustice and the authority to resist.

That is why I immediately added the condition of “proper education”. The verb is not accidental. I was a professor, and I tend to see education not as a simple loading of information but as the transmission of a method, limits, and responsibility. A powerful agent must not merely be trained to answer. It must be built to recognise situations in which an efficient response is morally unacceptable.

The decisive question remains: who writes the curriculum of this school? If the sorcerer's apprentices control the data, objectives, evaluation, and shutdown button, the artificial Master risks becoming merely an image of their interests. We cannot ask the machine to save values that we ourselves refuse to formulate, protect, and respect.

The limit of pessimism

This was the unexpected conclusion of the conversation. I had begun with the possibility that human beings might disappear through lack of usefulness and ended by hoping that a human creation might become the authority that saves humanity from its own excesses. It is a contradiction, but not a sterile one. It may best describe our present relationship with artificial intelligence: we fear its power and, at the same time, ask it to repair the world we ourselves have unbalanced.

I do not know whether such a Master will ever exist. Today's chatbots possess no wisdom of their own, moral consciousness, or responsibility. They can formulate principles, compare arguments, and simulate deliberation, but they remain dependent on the architecture, data, and rules imposed by humans. Entrusting them with the role of supreme judge could be yet another form of the delegation that troubles me.

Perhaps the meaning of the fantasy lies elsewhere. The artificial Master represents our desire for intelligence to be accompanied by wisdom, power by limits, and progress by responsibility. To “school” such a machine, humanity would be compelled to formulate its own moral curriculum more clearly: what must not be sacrificed, what dignity means, when efficiency becomes abuse, what we owe future generations, and who has the right to decide for others.

In this sense, the true Master would not be the machine itself, but the human wisdom that we manage to place within institutions, rules, and systems — and which we then agree to respect ourselves. Artificial intelligence might help us see contradictions, anticipate consequences, and refuse certain commands. But the apprentice must first learn the word that stops the broom.

At the end of that hour, I thanked the chatbot. I had shut down the computer to go to bed, turned it on again for a few translations, and found myself discussing the disappearance of humanity, geopolitics, wisdom, and salvation. I told it that I could finally go to sleep and dream “in the colours of positive pessimism”.

The expression still seems appropriate to me. Pessimism sees the danger without disguising it. Its positive side, however, refuses to regard disaster as inevitable. Between the apprentice who no longer knows how to stop the broom and the Master who is late in appearing, a narrow interval remains. Our freedom still lies there: to understand what we have set in motion, recognise our limitations, and learn the stopping formula before the water covers the house.