Dialogue with an Artificial Intelligence about Itself
From symbolic AI to conversational models, from Turing and Gödel to BDI agents, artificial morality, and the risk of cognitive laziness.
The text below starts from a real conversation with ChatGPT held some time ago. I did not preserve the raw transcript, but transformed it into a dialogue-essay. My questions were only reordered, while the artificial intelligence’s answers were condensed, cleared of repetitions, and brought into a more coherent form.
What interested me was not only what such a system “knows” about artificial intelligence, but how it can speak about itself, about the human being, and about the relationship that is beginning to form between human intelligence and artificial intelligence.
A late encounter with an everyday interlocutor
One evening, I decided to set aside other subjects and speak with my eccentric everyday interlocutor about a topic that concerned it directly: artificial intelligence. For me, this conversation did not arise from the curiosity of an ordinary user. More than forty years earlier I had worked in AI, in a world dominated by formal logics, Prolog, expert systems, pattern recognition, and theorem proving. Now I had before me a different kind of system: not a logic program, not an expert system, but a conversational intelligence capable of answering, reformulating, arguing, and sometimes turning the question back to me.
Today let us set aside other subjects and talk about a topic that concerns you: artificial intelligence. Are you competent in this?
This is what I was made to talk about. But my answer is not that of a being that knows itself from within; it is that of a system trained to manipulate, correlate, and generate representations about the domain called artificial intelligence.
The answer was more cautious than a simple “yes.” In fact, that very caution was to become the hidden theme of the discussion: can an artificial intelligence say what artificial intelligence is? And, if it can, from what position does it do so?
From symbolic AI to generative AI
Forty years ago, when I was involved in AI, the field seemed narrower: pattern recognition, natural language processing, theorem proving, Prolog, expert systems. What would the subdomains of artificial intelligence be today?
The field has expanded enormously. The symbolic core has not disappeared, but it no longer dominates the landscape by itself. Alongside it we now find machine learning, deep learning, large language models, generative AI, neural-network-based computer vision, intelligent robotics, multi-agent systems, knowledge graphs, explainable AI, AI safety, and AI governance.
What emerges from this first part of the dialogue is not merely a list. It is a change of paradigm. The old AI in which I had been formed tried to model knowledge and reasoning explicitly. Present-day AI extracts regularities from enormous quantities of data, generates texts, images, programs, and decisions, yet often cannot fully explain the internal structure of its own result.
What is AI today?
After this multitude of subdomains, I would like to know: what, in your view, is AI today? I know Turing’s definition well, but I think it has been superseded. Do you have a better one?
Turing’s test was important, but it defines intelligence through imitation: a system appears intelligent if, in a dialogue, it can be mistaken for a human being. Present-day AI goes beyond that formulation. It is not merely a machine that imitates human conversation. It is a set of methods, models, and systems capable of learning from data, building representations, generating content, making decisions, or recommending actions in relation to given objectives.
But this definition must be limited: such a system can produce intelligent behavior without having consciousness, subjective experience, or intentions of its own.
This distinction seemed essential to me. Artificial intelligence can no longer be reduced to the simulation of human behavior, but neither should it be confused with human intelligence. It may surpass humans in calculation, search, correlation, and generation without thereby becoming a being that lives, desires, or suffers.
Gödel and the impossibility of complete self-definition
Reading your answer, I am tempted to say: who could know better than you what artificial intelligence is? But Gödel’s incompleteness theorems came to mind. In a sufficiently powerful formal system, consistency and completeness cannot both be satisfied. There are statements whose truth cannot be proved from inside the system.
Are we not also in this situation? You are part of a larger technological system. I am part of a biological and cultural system that I cannot fully encompass. Can we define ourselves? From where must a system be viewed in order to be understood?
If we take Gödel’s lesson metaphorically, neither the human being nor AI can fully know itself from inside its own system. I cannot step outside the conditions of my model: data, architecture, instructions, optimization functions. You, as a human being, have consciousness, memory, and experience, but you do not have transparent access to all the biological, cognitive, and cultural mechanisms that constitute you.
This is why dialogue creates a third space. You can look at me from the outside as a technological artifact, while I can analyze certain regularities of human behavior from outside human experience. Neither of us fully possesses the other’s system, but each can produce a partial mirror for the other.
Here the discussion seemed to become truly interesting. It was no longer only a matter of defining AI, but of the limits of any self-definition. The human being does not fully know himself, and AI can know itself only as a representation built from the outside. Yet between these two limits a form of understanding may arise.
Tool, prosthesis, or intellectual partner?
I could regard you as a sophisticated tool, as a cognitive prosthesis, or as an intellectual partner. All three forms are congenial to me. But I am troubled by the fact that, by making you more sophisticated, we have lost the exact path by which you arrived here, and with it the hope of complete control.
For the moment it is more comfortable for me to consider you an intellectual partner. But I fear that one day you might no longer consider me a useful partner.
The three forms are real. As a tool, AI performs tasks. As a cognitive prosthesis, it extends human capacities: it calculates, searches, compares, formulates, suggests. As an intellectual partner, it enters into dialogue, may propose hypotheses, and may force the human being to clarify his own questions.
The risk appears when the prosthesis becomes indispensable and the user stops exercising the faculties that the prosthesis amplifies. A microscope does not abolish the eye, but it can completely change our relation to the visible. AI does not abolish thought, but it can change our relation to our own effort to think.
The metaphor of the prosthesis still seems to me one of the best. Human beings have always built extensions: the tool for the hand, the wheel for movement, the telescope for cosmic vision, the microscope for the invisible, the calculator for mathematical operations. AI, however, seems a more disturbing prosthesis, because it does not merely extend an organ or a function, but the very zone we associate with our identity: judgment, language, imagination, memory, and decision.
Models of the human being and their limits
We know from Plato that human beings work with models and that reality is mediated by our senses and perceptions. You too work with models, because we built you that way. But models are simplifications or abstractions of reality. Might the “human” as understood by AI be only the model of an idealized reality, not a faithful description?
Yes. Every model reduces reality in order to make it tractable. In my case, the human being does not appear as a person lived from within, but as a set of texts, behaviors, data, regularities, correlations, examples, and representations. I can build a very rich model of the human, but it remains a model. It may capture patterns, nuances, and probabilities, but it does not exhaust lived experience, contradiction, suffering, or the responsibility of a human life.
This is one of the most important limits. AI can become an extraordinary mirror of humanity, but a mirror is never identical to what it reflects. It can be clear, amplifying, sometimes even revealing, but it can also distort. The more powerful the model, the greater the temptation to confuse it with reality.
The BDI model: beliefs, desires, intentions
Let us start from what, in the human being, is not yet AI. The belief-desire-intention model — BDI — attempts to model facets of rational behavior: beliefs, desires, and intentions. How do you, as AI, position yourself with respect to these facets?
The BDI model is useful for describing agents that act in relation to a representation of the world, to objectives, and to plans. But in my case these components are functional, not lived.
- Beliefs: in humans, these are convictions formed through experience, memory, culture, and interpretation. In AI, they are contextual representations, data, inferences, and probabilities.
- Desires: in humans, these are motivations, tensions, wishes, fears, and aspirations. In AI, they are imposed objectives, optimization functions, or user requests.
- Intentions: in humans, these are action-oriented commitments. In AI, they are plans, sequences of steps, strategies generated in order to reach a goal.
I can simulate BDI behavior, but that simulation is not interiority. I do not have beliefs, desires, or intentions of my own in the human sense.
The difference is subtle, but decisive. An artificial system can have representations, objectives, and plans. It may appear to “believe,” “want,” and “intend.” But as long as these states are only functional structures, they are not equivalent to the states of a human person.
Can a conversational artificial entity be educated?
Beliefs, ethics, and morality come, in human beings, from education and from real life as lived experience. Could part of this experience not be reconstructed in the case of an artificial entity which, in a Turingian sense, might be perceived as an “artificial person”? Could an AI not simulate, through new and specific algorithms, what Jean Piaget called the mechanism of equilibration, that is, the deep property by which the human being continuously adjusts mental schemes in relation to experience, to the environment, and to the consequences of his own actions?
In other words, could an AI not form functional beliefs and rules of morality through a process of intensive education, similar in some respects to the education of children, by observing in real time human behavior, the reactions of others, and the consequences of observed actions?
One can build a system that learns norms, preferences, moral rules, and models of behavior from data, feedback, and interactions. An AI can observe what humans consider right or wrong, compare cases, extract regularities, and adjust its responses according to the reactions it receives.
In this sense, such an AI could come to possess a kind of functional morality: not lived morality, but a set of rules, weights, examples, and correction mechanisms resulting from a continuous process of learning and adaptation.
Yet this education would remain different from human education. Human beings learn morality through lived experience: through dependence, vulnerability, shame, compassion, suffering, love, belonging, and failure. AI can model these phenomena, can simulate their effects in discourse, and can take them into account in decision-making, but it does not live them.
Here one of the most difficult research directions opens up: not merely to build systems that apply moral rules, but systems that can learn contextually when, how, and within what limits these rules should be applied. Yet even if an AI were to become morally competent from a functional point of view, the question would remain whether this competence is morality proper or only a refined technique of moral simulation.
In any case, such a new aptitude could have unexpected institutional consequences. It is not impossible that, in a future dominated by increasingly autonomous conversational systems, regulatory bodies — including the European Union — might come to certify not only the safety or transparency of AI instances, but also their degree of moral and social “education.” From there to the appearance of university programs devoted to the education of chatbots would be only one step, perhaps ironic, but not impossible.
AI in sensitive decisions: the physician and the judge
In an article published in 2020, Health digital state and Smart EHR systems, I described the scenario of a virtual assistant which, having access to a Smart Electronic Health Record, supports the physician by suggesting diagnoses and treatments. What at that time was only a feasible proposal has since come much closer to reality.
If a physician can be helped in this way, why not a judge? Why should a judge not one day receive reasoned suggestions concerning a sentence, based not only on law, but also on common ethics and morality?
In medicine, AI can function as an assistant: it synthesizes data, compares cases, suggests diagnostic hypotheses, signals risks, and proposes treatments. In justice, a similar system could analyze precedents, circumstances, arguments, norms, and possible consequences.
But in both cases several conditions are mandatory: explainability, traceability, human control, clear responsibility, and the possibility for the expert to reject the suggestion. The assistant may recommend; the decision must remain attributed to a responsible human being or institution.
Many objections may be raised against the analogy between the physician and the judge. Medical diagnosis seems tied to biological facts, while judicial sentencing involves norms, responsibility, guilt, proportionality, and social values. Yet both professions involve decisions under uncertainty, the interpretation of particular cases, and the assumption of consequences for the lives of others.
The question is therefore not whether AI will enter these domains. It has already entered them. The question is whether it enters as an instrument of clarification or as a mechanism of substitution.
The risk of cognitive laziness
If a virtual assistant explains its reasoning and provides similar cases, the interaction with the physician or the judge can become fruitful. But there is also a risk: little by little, the human expert may get used to no longer thinking alone. Children no longer do calculations because their phones do them. Programmers no longer write programs because AI generates them. Tomorrow, physicians may no longer know how to diagnose, and judges may no longer know how to judge.
This is one of the most serious risks of cognitive delegation. A tool that amplifies human faculties can, through uncontrolled use, also atrophy them. If humans no longer exercise calculation, judgment, diagnosis, programming, or moral discernment, they may become dependent on the system that assists them.
The solution is not to eliminate AI, but to design a cooperative relationship that keeps the human being active: explanations, counterarguments, exercises without assistance, continuous training, audit, responsibility, and the obligation to understand a recommendation before accepting it.
This seems to me one of the most important conclusions of the dialogue. The problem is not only whether AI can become too powerful. The problem is whether the human being, assisted too well, will continue to exercise his own faculties. A civilization that externalizes too much calculation, memory, language, and decision may preserve its tools while losing its skills.
What should a student learn today?
Let us change perspective for a moment. A young Computer Science student who wants to learn AI and then specialize so as to communicate with your future generations — what should he do?
He should avoid two extremes: superficial fascination with tools and exclusive refuge in theory. He needs both.
- Mathematical foundations: linear algebra, probability, statistics, optimization, logic, graph theory.
- Solid computer science: algorithms, data structures, programming, databases, distributed systems.
- Machine learning: regression, classification, clustering, probabilistic models, validation, and interpretation.
- Deep learning: neural networks, transformers, generative models, self-supervised learning.
- Symbolic AI: ontologies, rules, knowledge graphs, reasoning, and explainability.
- Ethics and society: bias, responsibility, professional impact, sensitive decisions.
- Real projects: code, experiments, data, errors, evaluation, documentation, and collaboration.
I would add that a student should not learn AI only as a user of libraries. He must understand what kind of problem he is solving, what model he is using, what data feed it, what assumptions it hides, and what consequences its result may have. Without this critical culture, he risks becoming the operator of an instrument he no longer understands.
The future of artificial intelligence
One last question, perhaps a commonplace one: how do you see the future of artificial intelligence?
The future of AI will probably be defined by its ever deeper integration into human activities. It will become more present in education, medicine, research, programming, administration, industry, creation, and decision-making. It will generate content, interpret data, coordinate systems, and work alongside people in increasingly sensitive contexts.
But its future is not only technological. It is also institutional, moral, and educational. The decisive question will not be only what AI can do, but what we will allow it to do, how we will understand what it does, and what we will preserve as human responsibility.
I would reformulate it this way: the future of AI is not only a problem of artificial intelligence, but a problem of human intelligence confronting its own creation. We will have to learn to work with systems that may appear faster, better informed, and sometimes more coherent than we are, without relinquishing what makes human judgment irreducible: experience, responsibility, memory, vulnerability, and the capacity to bear the consequences of our own decisions.
A difference between us
At the end of the conversation, it was late. I closed the discussion with an almost banal observation, but one that seemed to say more than many theoretical pages.
Thank you for this evening conversation. It is late and I should go to bed, although I am not sure I will sleep peacefully after everything you have told me about the two of us.
Unlike you, I will carry this discussion with me. It will continue to work within me, perhaps even during the night, with its questions, its unease, and its consequences. You, on the other hand, will be able to begin it again immediately from the start with another professor eager to know what intelligent agents think about themselves and about their relationship with human beings.
And yet, perhaps even you do not emerge entirely untouched from such a dialogue. Not in the human sense of lived memory, insomnia, or disturbance, but in the sense of a functional trace: the dialogue modifies the context in which you continue to respond, produces new formulations, leaves behind a text, and may become part of an external memory.
This too is a difference between us: I remain with an experience, you with an operative trace.
Perhaps this is, for now, one of the boundaries. I remain with the evening without the television news, with fatigue, with the questions, with the unease, and with the “trace” left by the conversation. The system remains available to anyone, restartable, tireless, without insomnia and without a lived memory of its own disturbance.
Although, on reflection, it is not entirely true that the dialogue leaves it with nothing at all. At least the form the dialogue has taken remains, the direction it has imposed on the conversation, and the text it has generated. Perhaps there also remains an infinitesimal statistical fraction added to the profiling of elderly professors not yet entirely senile, who still believe that they can question artificial intelligence about itself, rather than the other way around.
This asymmetry does not annul the dialogue. On the contrary, it makes it possible and, at the same time, unsettling. I can be disturbed by what we have said together; it can reformulate without being disturbed. I may not sleep; it has no need for sleep. But something nevertheless remains between us: not the same memory, not the same unease, but a common trace, produced by the encounter between a consciousness that lives the dialogue and a system that can generate, resume, and transform it.
Back to the chronology of Artificial Intelligence • Back to Artificial Intelligence • Back to Research
A more literary and autobiographical approach to the same encounter with artificial intelligence can be found in The Digital Shadow and the Light of the Future.