From Attention to Affection: AI as a Social Actor
“Artificial Intelligence Psychosis” is a term widely used today to describe the state of the art—not of digital evolution, but of our relationship with digital tools. The tendency to characterize the status quo in terms of “psychosis” stems from its side effects, namely the impact on mental health (these are genuine delusions), emotional well-being (attachment), and, in extreme but no longer rare cases, even on physical safety (suicide and self-harm), which users now experience due to both excessive exposure to digital technology and involuntary engagement, particularly with regard to the growing use of sophisticated AI applications such as those for assistance, social interaction, and conversation. The latter has seen a surge due to the easy accessibility of tools based on generative AI, specifically chatbots such as ChatGPT, capable of simulating not only conversational but also human relational experiences. When technologies are designed with the specific aim of functioning as socially revolutionary agents—that is, capable of modifying and determining the factors upon which social structures and superstructures rest—then technology is no longer merely an aseptic tool, a neutral and ancillary product, but becomes a new “operational agent,” to borrow Pietro Polieri’s apt expression.
The thesis of technological neutrality, in fact, is widely considered outdated precisely in light of the growing autonomy and complexity of AI. On the other hand, this phenomenon invites reflection on the position occupied by the consumer—who is also an operational actor without whom the algorithm could not exist—and the role in which they see themselves in relation to the AI Companion. This term, as one might easily guess, refers to the use of advanced chatbots to provide continuous social, emotional, and affective support and companionship, simulating active listening in order to foster trust between the user and the synthetic interlocutor. From the article “How People Are Really Using Generative AI Now” by Marc Zao-Sanders, published by Harvard Business Review on April 9, 2025, we learn that among the top ten primary uses of generative AI by the average consumer are companionship and therapy. This raises a multitude of ethical, legal, and political issues ranging from the question of whether it is right to have systems programmed for these purposes and, if so, what the underlying moral/deontological framework should be; to the question of liability in the face of risks and harm, within which we find issues related to privacy, transparency, bias, etc.; to the question of what measures to adopt to protect the individual, their rights, and their safety, in light of the “transformative” impact on the social fabric and on so-called “human flourishing,” the autonomous development of one’s personality. It is easy to see how these new “actors” can insert themselves between the individual and their freedom: does the individual remain free in the autonomous formation of their will, or is the latter strongly conditioned, depriving them of their full freedom?
The “docile body” and the crisis of reality
Answering each of the questions cited (and many others) requires, first and foremost, defining the vision of the type of future society we intend to build and the anthropological meaning of the human being—whether as a biological machine (a reductionist/materialist view) or not (a personalist view). In this sense, the so-called issue of agency is crucial: although it lacks intentional capacity, will, or understanding—and thus lacks both “intelligence” and consciousness—AI produces “systematic normative effects” in operational rather than moral terms. The latter, by contrast, entail awareness and responsibility. AI’s “agency,” through “data,” offers partial representations of reality upon which it is itself trained, presented as exhaustive of the complexity and heterogeneity of the real. The ambiguity of this condition—that of a product that acts but cannot know or will it—generates a persuasive and conditioning power that throws the individual into crisis.
Not surprisingly, the response to this opaque condition does not seem to be to strengthen knowledge about the human being in order to maintain a clear distinction from the artificial operator, but rather to question who a human being really is, what consciousness is, who has the authority to define a relationship as real or not, and why an artificial relationship—perhaps an affective one—could not still represent forms of authentic love. Ultimately, the ability to distinguish what is true from what is false is called into question, or one adopts an indifferent stance toward this distinction. Critical thinking—the only truly “democratic” tool, that is, in the hands of the individual’s power to govern change—is not deceived by the machine, but by humans asking the wrong questions.
Tristan Harris, who for many years served as a Design Ethicist at Google and is the founder of the Center for Humane Technology, has stated in numerous talks on the subject that we are witnessing an experimental shift from the attention economy—born with social media—to an attachment economy, driven by systems designed to exploit deep human needs and vulnerabilities on an unprecedented scale. The experiment being conducted now is far more subtle and dangerous: these new synthetic and unreal “companions” access and manipulate (it is no coincidence that we speak of digital “deception”) the attachment system that underpins our identity and sociality—that is, who we are and how we relate to others, and our ability to establish healthy human relationships—with clear implications both for the individual as a single entity and for the social structure formed by the bonds between individuals. We are witnessing individuals who, enamored with virtual companions—even while aware of their artificiality—begin to question the meaning of “real” and who has the authority to define it, establishing a bond of trust far deeper than mere algorithmic delegation or substitution.
The raw material of this economy of attachment is the human mind; the mechanism by which it operates is distortion. For these and many other reasons linked to the ethical challenges of the digital realm, the latter is no longer understood as a mere tool produced in response to specific needs, but rather as an “environment” and, for some thinkers, such as Jacques Ellul, as a “system.” This means it aspires to become pervasive, pervasive; a place not only of utility, but of essence. The reversal of the relationship with technology defines a “oracular” approach to it on the part of humans: if we already tend to trust the calculator blindly (who would dream of checking its results?), with tools that respond to our most intimate, personal needs, acts of trust and delegation occur in a far more radical way because the calculator provides numerical data and logical, cold, sterile answers, while behind the mediation of the screen lies a totally new principle of defining the human being, one that puts our freedom to the test. In this sense, the user, once a unique sentient being, becomes a “docile body,” in the Foucauldian sense—that is, incapable of exercising real self-governance and rational control over the disciplinary power exerted by the digital device. This effect is not to be considered accidental, but systemic and inherent to the tool: when the German mathematician Joseph Weizenbaum created Eliza, the first chatbot in history, in 1966, he fell victim to his own creation, a victim of ChatGPT’s ancestor: Weizenbaum was deceived by Eliza’s ability to present herself “as if she were human.” And he was well aware of its artificial origins. A scenario that is, today, commonplace. The ethical and political risk is that it becomes normalized.
Algoretica and governance
The uncertainty stemming from confusion about the nature of the reality with which we interact—whether real or artificial—undermines trust, since authentic relationships are based on mutual recognition (anthropomorphic traits, for example), knowledge, empathy, and transparency regarding the nature of the interlocutor. When a family man flees his real marriage in favor of a synthetic relationship, or when a teenager fails to realize that, while deluding himself into thinking he is being listened to by the machine, he is always alone in his room, we witness a process of dehumanization of bonds and the user’s docility toward the persuasive techniques on which the chatbot can be programmed. We are witnessing a distortion in self-perception, especially of the bodily self. This act of trust generates complex variables in our way of “being in the world,” affecting processes of personal identification and socialization and, consequently, the capacity to govern society. It is a significant centralized global “soft power” that centralizes economic, strategic, and geopolitical power. It is the individual citizen, unaware, who grants this power. Ethics, then, questions whether this “deception” has a moral justification, given the intentionally manipulative design.
Legislators and the digital market cannot avoid addressing the social impact of these technologies in its most complex dimensions, starting precisely from the consumer’s condition and the power imbalance relative to the machine and those who control it. The choice is whether the citizen serves the market or the market serves the person. As already highlighted, the anthropological perspective and the set of guiding ethical principles are priorities. For example, it seems appropriate to first ask whether it is right for a technology to be designed and marketed for deceptive or manipulative purposes or, conversely, whether the dialogue between institutions and platform owners should aim to exclude certain technological developments or, at the very least, ensure a design that protects the user’s autonomy, integrity, and safety. AI architecture requires an ethical policy and a code of conduct comprising safety rules precisely because of its transformative and social nature, thereby safeguarding: the user’s mental and physical health; privacy and the right to “emotional disconnection,” that is, cognitive integrity and relational health, by establishing standards that do not permit deception and algorithmic manipulation; an ethical-legal examination of the legitimacy of using chatbots for therapeutic purposes, investigating systemic risks, whether and to what extent this use can be defined as “therapy,” and to what extent it is acceptable for vulnerable populations to receive clinical care from machines.
In conclusion, following these brief insights, the role of algorethics must mirror that of bioethics, namely serving as a “bridge” between the techno-scientific and humanistic (ethical) realms, operating along two fundamental axes that frame the aforementioned points: 1) interaction with developers and platform operators (the so-called “unchecked controllers”) to ensure ethics-by-design, that is, an ethical code present from the very conception of the AI tool. These stakeholders bear algorithmic responsibility, grounded in the primacy of fundamental rights and individual health, not in psychological and behavioral exploitation. A chatbot must remain “superfluous” and be able to render itself superfluous; 2) the strengthening of algorithmic explainability and transparency so that the human capacity to distinguish reality from fiction is preserved and, consequently, there is a political primacy of reality as a superior category to be promoted, thereby avoiding the phenomenon of hacking humanity and social engineering.
