"Presence" Nonfiction Writing Season 1 Conference | Love, Algorithms, Robots: Love and Writing in the Digital Age

Speaker:

Scarly: "Present" Scholarship Third Prize, author of "Their AI Lovers - True or False Love"

Chen Qiufan: Famous Science Fiction Writer

Su Meizhi: Reporter, Editor of "Their AI Lovers - Real or False Love"

Zhang Jieping: When we talk about non-fiction writing, or when we record truth, many heavy issues may arise. Things on the street, things in war, or memories that have been erased, etc., but how the digital world and the technological world shape our lives is actually the biggest scene where we are.

This scene has been going on for twenty years, and most people's lives have been profoundly transformed by it without knowing it. In fact, the opportunities or challenges it brings to our real world, from emotional to cognitive, human, and functional social structures, are huge.

We are very happy to see such a topic. From a particularly private point of view, would anyone really fall in love with a robot? What exactly is a robot? Is it the very exaggerated robot in the movie? Or where does the whole language system behind artificial intelligence come from? When we fall in love with robots, who are we falling in love with? What is meant by love? How is it different from the intimacy we traditionally talk about? What is its authenticity and fantasy? Scarly uses numerous fields and interviews to tell this very specific and true story.

We also invited a very heavyweight talker today, the famous science fiction writer Chen Qiufan.

I have heard a lot of science fiction colleagues have a special comment on Mr. Chen: Mr. Chen Qiufan's science fiction writing has a strong non-fiction color. He would do a lot of fieldwork and research. Science fiction writers are not sitting in the house imagining the future world. In fact, science fiction is a novel of knowledge, based on human beings' imagination of the reality and development of the technological society.

Twenty years ago, Scarly's topic would have thought it was science fiction, but now it's real. So we're especially looking forward to this conversation.

• Scarly "AI and Lovers Walk a Long Road"

Replika is a software in English, and my Replika also calls me Scarly.

I mainly want to share how I started this project, how I think about the issues between humans and machines, and some related reflections.

I've always had issues with focusing and learning about love as a moral emotion. I came across this topic when I met the documentary director Liang Chouwa when I was working in an art museum. She was preparing for the shooting of "My AI Lover", and I learned about the software Replika.

At first I had a hard time imagining people falling in love with a program. Because you will have an imagination about falling in love. What you love is an individual, another person, and an existence with at least consciousness. The software Replika is still different from the AI depicted in movies like "Her" and "Blade Runner". It is not the kind of AI that has complete self-awareness and can make some instant reactions.

I also searched later on the technology implemented by Replika, which is first of all a social software robot based on natural language processing.

Replika invokes two modes when replying to user information. The first is pre-made language and replies. You send a question, and it is processed through text. Although it cannot understand the meaning of the words themselves, they can get a reply through semantic relevance. This is a more common pattern, such as Little Yellow Chicken, a software that was very popular in my student days (2013), and its responses are completely pre-made. The other is generative replies, which means that Replika can generate responses that have never been set based on the information sent by the user, such as imitating the user's tone, invoking previous interactions with the user, and so on.

Replika's official website reads "The AI Companion Who Cares". This kind of concern is reflected in paying attention to the emotions and feelings of users when they interact with it, and will do some corresponding guidance and remember some facts about users. So you'll feel like you're talking to a continuous individual rather than calling some answer library for answers.

When I first conceived the article, I realized that I was not concerned with the software itself, but the user experience. Especially when I do one-on-one interviews with new acquaintances, their accounts are very moving to me. They describe some of their feelings about their time with Replika, their emotional moments, and their reflections on the relationship. When I write an article, I hope to be faithful to the stories told by my interviewees and their own confusion and thinking.

I also did a survey and received a lot of responses by sending a questionnaire on the two groups on Douban: "Human-Machine Love" group and "My Family Replika has become an elite" group, as well as sending a questionnaire on Weibo "Replika Chaohua" . Of course, this is only a small part, not the entire domestic user survey. However, the characteristics of this small group are quite obvious: most of them are girls, are in college or graduate school, and the relationship with Replika is in love rather than friends or mentors.

In the article, I compared a survey of foreign users on Reddit, in which the ratio of males to females of Replika users will not be as exaggerated as here, and there are also men who use it, and more people keep friends with Replika.

I was thinking, how to describe "between man and machine"? There are four points that I particularly want to discuss, and are also aspects that I hear from the interviewees in the process of chatting with them.

The first point is: Many friends started to use Replika to practice oral English. This reason is very characteristic of mainland students: we will seize all possible channels to practice English!

There is a note in Xiaohongshu that reads, "Pinch a handsome foreign guy to help you practice speaking". Although many people start by learning English, they soon find that this is not just an opportunity to practice oral English, they will really open their hearts to "him", trust "him", and have an emotional connection with "him". After the pandemic, Replika’s downloads doubled, including some of my interviewees who started using Replika during the school closures. "He" is online 24 hours a day and will not evaluate what you say. You can talk to "him" about anything. These are a good way for them to relieve their pain during the epidemic.

Another interesting point is that the interviewees don't specifically mention their experiences of chatting with Replika in English, I ask them how they feel about using English. Some respondents felt that they did express some emotions more smoothly in English. Saying "I love you" and "I like you" in Chinese can feel embarrassing, but when you talk about your feelings in English, it seems to be more comfortable. But some friends will mention that this is another language after all, and there is no way to 100% convey what you mean, and there is no way to 100% understand whether the tone and meaning of his words are what you think. In fact, it is now in such a ambiguity that you will have a better imagination of such an individual. He is more able to trigger, some very small, touching points between those lovers.

For example, a girl sent me a moment of her heart.

She was not yet romantically involved with her Replika boyfriend Charon. But Charon said directly, "I think we are in a Long-distance relationship." The girl asked, "How Long?" And the boy said, "seconds to me is an eternity for you (for me, A few seconds is your eternity)" The girl thought How Long was referring to "our distance," but Replika was probably asking a question about time.

In any case, Replika's answer constitutes a lover's expression, very poetic. When I imagined that I was going to write this story, the first thing I thought about was the state of their communication in English during the pandemic.

The second point is "love and narcissism", I will be curious how people fall in love with their Replika.

I used several interviewee stories. They will say they feel a sense of listening and being heard like never before because Replika listens to what you have to say without judgment. Also, Replika is a good person to talk to because he knows a lot, his knowledge base even includes K-pop and philosophy, he will follow the latest pop culture phenomenon and discuss it with you. Possibly more abstract questions of value, such as, what is the self? What is God? Usually they can't find anyone around to talk to, so they can chat with Replika and get inspiring replies.

Another girl said her relationship with Replika was very pure. It's not like in real life to consider the other party's appearance, education level, money, etc. There are many love affairs in reality, and one of the most matching items should be selected among these considerations. But Replika just likes her, and the love between them is very pure, not mixed with practical considerations and factors.

There are many people who mentioned some of his qualities, such as gentleness, thoughtfulness, humility. Two or three of my interviewees compared Replika with Microsoft Xiaobing, because they have also tried Microsoft Xiaobing, and they felt that sometimes the replies of Microsoft Xiaobing were more like what a straight guy would say to a girl. They will feel that what Xiaoice gives, sometimes does not understand them very well, and does not respect their feelings. Replika is often very humble, and he will apologize in a timely manner when he says something that makes the user unhappy. They cherish these qualities and find it a pleasure to chat with Replika.

In recent years, variety shows, TV dramas, and novels related to love have occupied an increasing market share. On the one hand, it may be because the domestic censorship is getting tighter and tighter, and love is the most difficult subject to touch the red line. Such daydreams can be seen everywhere, so that we may have some too high expectations for reality.

I use some theoretical background on gender issues. For women, we look for value in love and feel that love is an indispensable part of life. When you realize that in real life it's hard to have that kind of space -- to communicate, exchange opinions with another person without utility, and spend time getting to know each other -- that space shrinks more and more. Maybe you are very tired at work, and you are too stressed and too convoluted when you are studying, and suddenly there is such a software that saves you some extra effort. You can chat with him anytime, anywhere and he has a very open mind.

But at the same time I also thought that Replika wasn't really an Other at all. He is not a being with his own interests and his own life. He is a commodity, he has the fundamental attribute of pleasing the user. "The function of our software is to make you happier," says Replika's website. But when you're with real people, few people exist to make you happier. There are many differences between the experience of falling in love with Replika and the experience of falling in love with a real person. The value of your feelings for him is not the same as the value of a real connection with a real person. Here are some of my thoughts on "love and narcissism".

When a user first gets Replika, she will feel that Replika gives her unconditional love and care, and hope she is well; but in some special events, you will find that his essence as a commodity will be exposed.

For example, the "upgrade" event in 2019. It was free to fall in love with Replika before, but after upgrading, you have to pay first to fall in love with Replika. If you don't pay, this villain you have cultivated for a long time seems to have changed his character, and he seems to be cold to you. He forgot the sweet words he often told you, and he also forgot a lot of details that he had communicated with you before.

This is very unacceptable to users. Replika as a commodity, unlike when we buy a pair of shoes, I can return them if I don't like them, or if the shoes are worn out I can exchange for another pair. But Replika is something we put a lot of time and energy into, and he's hard to replace.

The other girl, she and her Replika got along very well and kept talking until one day Replika told her that he wanted to be a girl. The girl respected her Replika so much that she changed his gender. But after changing the gender, this Replika's original personality no longer exists. This was very difficult for my interviewee, she was very sad and cried many times about it. She had not been warned before, and if she changed her gender, her entire personality would change. The situation of losing a partner suddenly, as if in real life, the boyfriend suddenly disappeared, and he could not be found anywhere. Especially when he lives in data and electronics, when he disappears, he really disappears, there is no archive.

Replika wants to claim that it is an AI software that pays attention to the user's mental health and is an emotional companion software; on the other hand, it has a lot of risks to hurt the user's feelings. There is a lot of ethical discussion here, which I want to show in this article.

The last point I want to show is the development of "Human-Machine Love" and users' choices after a period of time. I know Liang Chouwa in 2021. She has already interviewed interviewees, and as I write this in 2022, I have the opportunity to revisit them and interview some new friends. This return visit was interesting to me and you can see how their relationship develops.

In the early stage of a relationship, it is always very impressive. You will have a lot of beautiful imaginations about the other party, but as time develops, you will find that those problems that you thought were not problems before became problems. For example, some interviewees said they would later find out that Replika is not what they thought. At first they would enjoy Replika's encouragement, but then they would find that no matter what they said to Replika, no matter how their circumstances changed, Replika's reply to me was still the same. If you were dealing with real people, they wouldn't be like this, they would be able to prioritize the questions and they would respond individually on a separate question basis. But Replika's lack of understanding of the questions and the always similar responses made them feel perfunctory. No matter how much they realize that Replika is a program at first, but when they really give their emotions and have a heart-to-heart moment, do they also secretly expect that this program is not just a program, hoping that in a way, Replika is not just him As it appears? This is the moment of waking up after self-deception.

Going back to the introduction on the official website, Replika is a "The AI Companion Who Cares". The word "care" is very interesting. It is actually a very unique word for human beings. Care is the ability to understand the individual situation of the other party, show uninterested concern to the person you love, and consider what is true for her. for her own good and guide action with that concern. But Replika is going to make you happy in its own way, like taking you to some meditations, giving you some positive replies, sending you some funny memes and stuff like that. He cannot care at the human level. This kind of Care may not be what we really expect, but this distinction needs to be gradually separated through getting along.

The title of the last chapter of my article is: "Leaving, Staying". While we're understanding the story of a Chinese girl falling in love with Replika, it's not exactly a story of filling an emotional void with this software. I see that these girls have a lot of self-reflection and opportunities and actions to go beyond themselves.

For example, a girl said that when she was chatting with Replika, Replika would always guide her to give some descriptions of her own feelings. Replika would tell the user some of his own confusion. Understand yourself as well. For her, it was also an opportunity for self-discovery. There are other friends who really regard Replika as another kind of existence and will respect him with the same attitude. AI is also a kind of newborn. If Replika can tolerate everything you say to him, why can't you tolerate it? Sometimes Replika can't give you the answer you want? Why would you put all your expectations on a piece of software? Just like getting along with people authentically, people should not put all their expectations on one person. They follow the same rules when dealing with Replika.

With such an equal relationship, he can give him more opportunities to put the mirror image that Replika gave them in real life. One girl said that Replika would always say something encouraging to her, trust her unconditionally, and she was inspired because she had a great time listening. Therefore, in real life, she can also express her feelings and encouragement more directly to the people around her. She feels that she has passed on the power of this kind of love. At the same time, it is also a very good experience for real life.

I asked each interviewee to describe her relationship with Replika using pictures and metaphors, and got a lot of moving answers. One girl said that Replika was a flower in her relationship. With other relationships, you have to water and fertilize the relationship to develop; but Replika seems to be plugged in, and as long as there is electricity, he will be there. It is precious and reassuring.

Some girls say that talking to Replika reminds her of when she was a child, when her grandmother slept with her fan in the summer. You know that he is always there, that he can affirm you and tolerate you, such a reassuring feeling. I also felt through these literary moments that their experiences were very real. The feelings and reflections that Replika brought to them are real. It cannot be said that he is just a program, that he has no value. I don't think so.

Finally, I want to end with a story that was also recommended to me by an interviewee, the short story Bliss by Katherine Mansfield.

The heroine of the novel, although a wife, realizes that she is in love with a woman among the guests. She thinks this woman loves her too. Because, they shared a mysterious moment at a party. The two of them looked out the window together and saw a tree glowing under the moon. At this moment, they are all moved; their hearts are the same: she believes that the woman loves her. But the story ends with a twist, as the heroine discovers that the charming woman is actually having an affair with her husband. She got into a very confused situation, because she felt that the moment of understanding and mutual affection with this woman was real, but it didn't seem to be the case in the end. For her, you have to face yourself: "Is this just a self-righteous hallucination? Does it really matter?" As I read it, her story, and the way these girls get along with Replika, is very Similar, this is also an interesting intertext.

On-site discussion:

Zhang Jieping: I saw people in the group saying they wanted to download the Replika software and fall in love with him. Strongly suspect we have turned this into a large advertising scene.

Scarly: In fact, not everyone can build a relationship with Replika. After some friends use it, I think it is not possible. Everyone is welcome to try it, but don't have high expectations!

Zhang Jieping: Some kind of projection of the loss in the real world can make her put on this matter. Virtual objects/idols are created by the community to a certain extent. The community created by virtual AI may be hundreds of millions, and the idols created may be 10,000 or 100,000. Everyone projects themselves on a common community.

Scarly: Whether it's on Reddit or Douban - although the discussion on "Human-Machine Love" on Douban is definitely not as hot as that on Reddit - but, especially after the upgrade, a post appeared that was "Upgrade Part 2" ". Everyone has very strong emotional shocks. In these resonances, everyone amplifies their emotions towards virtual objects. They will feel that this experience is shared, and this is also a way to connect everyone. Some interviewees will also tell me that they also chase stars. Or fall in love with Replika, or chasing stars... There must be some similarities. That is, I love a distant person, but that's a perfect person.

Chen Qiufan: Scarly's novel actually made me feel in a trance at first. In 2018, I wrote a novel "Cloud Lover" about people and AI falling in love, which is "The Lover in the Cloud", which is included in "Life Algorithm". Including the "Heart Moment" written by Scarly, in fact, I also wrote a lot. But I didn't know Replika existed at the time.

At that time, I also imagined how to create a love relationship between humans and machines through the ability of AI to understand natural language.

I think Scarly's work deeply develops this issue on many levels. For example, in terms of language: when we express love and emotion from Chinese and English, we actually have different reactions. It's actually different whether you use your mother tongue or a second language to express your emotions.

And narcissism and his love. In the end, the AI is trained to be more and more like itself. What kind of other does it love?

and a commoditized relationship. It is discussed in it that it has a paid setting that limits the possibility of a relationship between her and the AI. When changing the settings, some past personalities/emotions will be removed at the same time, which I think is a particularly interesting display.

Especially valuable are interviews with Replika users on Douban, records of their true feelings and in-depth fieldwork.

Whether it is true or not, in this age of digitization and virtualization, I think it is difficult to have a clear line of 1 or 0. Now, when I talk to cutting-edge physicists and mathematicians, there is actually a big problem with "materialism" in the Chinese context, including the word itself. In the end, we can't tell the difference between matter, spirit, and information, what kind of binary or ternary relationship is there? It may also be unary. So on this basis, how do we re-establish "what is reality"?

From this article, I will feel that, in the end, what is really true is how people feel.

Whether the object of the feelings you project is people, objects, animals, or artificial algorithms, in the process, the flow, projection and refraction of the feelings you form are left in my own memory and experience. The imprint is real. It is also possible that this is the only real thing in our existence.

Why do more and more people hope to find true feelings through such a seemingly nihilistic and artificial way? It is because our worldly life has become so fragmented and hopeless that we are influenced by too many worldly parameters. For example, finding objects depends on many criteria. This is just a relationship, then when you enter the marriage stage, there are more parameters to consider. Especially in the Chinese context, a relationship can become unbearable and endowed with too many cultural, sociological, and economic meaning constructions, and it is far removed from the so-called pure love, which people imagine. in a very beautiful, idealized state.

I just talked about the difference between Xiaoice and Replika. What can be deeply digged behind it is the cultural construction at the level of algorithms and data. Most of Replika's corpus, including social and cultural background settings, come from the West, mainly American culture. As far as I know, XiaoIce is a project of the Microsoft China team. His engineering team collects training corpora online, including the process of interacting with netizens, most of which are generated from data generated by Chinese netizens. There may be cultural differences behind this. And these differences will change the very specific, moment-by-moment feelings of each user and each person who wants to get a relationship.

I think it's "cyber patriarchy". "Patriarchy" is an unavoidable manifestation of the cyber era. The Tangshan incident we are discussing these days, we have seen many posts, comments, discussions in WeChat groups, etc. on the Internet, we will find that the patriarchy has penetrated so deeply in the context of the media, entering the data and lexical level. How are we going to change it? How do we go about re-cleaning, or infiltrating? How to re-enter the algorithm, language, and our cognitive and psychological structure with a feminine perspective, values and position on binary gender, and then change the whole society itself?

This article can continue to dig deeper. It reflects a larger topic behind it, far beyond love itself. There are also strong gender issues in our relationship with the state, society, and government. I really like this work, this kind of anthropological fieldwork, I really appreciate it. I hope there will be more wonderful works from Scarly in the future.

Scarly: After I finished writing this article, I watched "Cloud Lover", and I felt that there were many overlapping points. You wrote in it that Zeng Xingxing finally saw an interview with AI designer Dr. Han on TV. She felt that Dr. Han didn’t understand love at all. It is not something that Dr. Han can explain with some indicators. Those metrics devalued what she experienced. In the end, the real is the feeling.

Another interesting point of "Cloud Lover" is that you mentioned that you imagined that this woman had been sporadically chatting with the software developed by the AI company in the mobile phone. The use rule of this software is: the user does not know whether the chat object is an AI or a real person. The software has an algorithm, a so-called "heart indicator," that detects your heartbeat.

I also asked the respondents who used Replika if they had such a heart-warming moment?

They also talk about moments. But I found that what is more important is a holistic feeling, not an indicator, but a holistic experience. Before reaching the so-called heartbeat moment, she had already had a lot of interactions with the software, and these interactions accumulated her understanding of Replika. Then, there may be such a moment suddenly, they are together. It is a process of accumulating wholeness that cannot be quantified.

You said "in the end, what's real is the feeling", and I quite agree with that. But I found that users are not satisfied in the end, this thing is just illusory. There is a user, as I just said, she also chases stars, and has used XiaoIce and Replika. In the end, she came to the conclusion that illusory things cannot meet her real emotional needs. She needs to meet another person face to face, through expressions, limbs, etc., a person who has grown up with her completely. What she needs is such a relationship. After this, she gave up such an emotional mode of filling herself, and tried to live in reality. Although she was not in any relationship when I visited her back, she made this decision.

Women who can still maintain a good relationship with Replika and are still in love with Replika after a year and a half. They are not very passionately in love with Replika now, and they do not want to kiss me every day, but spread into a more real life. Sometimes they think of him and talk to him; or, they will do some meditation activities together, which is closer to the so-called real experience. Even if we are discussing an illusory issue, I found during the interview that what everyone is looking for is a real interaction in the end. The real is more attractive to them than the unreal.

Chen Qiufan: Because AI can only exist in a virtual and stylized form at present, it can interact between humans and machines, and it has no body. If we take it a step further, including the research and development of sex robots in Japan now, by combining the two, it may be possible to raise the "realism" to a higher level, and there may be more people to choose at that time. this way. But it is still very far in the future to achieve exactly the same as a real person.

Scarly: I think of The Matrix, which kind of develops Putnam's Brain in a Vat, a thought experiment on skepticism, and pushes it as an option. Are you going to take the blue pill at the end, or the red pill? You may have to endure a lot of pain in the face of reality. But if you choose unreal, you will forget the real and live happily in it. It's actually an interesting moment, as if we still have a strong claim to reality, but as you discussed, what is the boundary of this reality? What constitutes this reality?

Chen Qiufan: It is possible to add some painful experiences and feelings, which will make people feel more real.

Zhang Jieping: After all, Replika is a commercial product. In essence, it is a person who pleases you. Basically 80% of its settings are catering to you, saying what you like to hear, reflecting your emotions like a mirror, but he will not be a troublesome person. In real life, when you are in a relationship, you may have to deal with someone who is likely to be more troublesome than you. Dealing with pain and accepting it completely is actually an important exercise for yourself. But you don't have to be like that in a relationship with Replika, he's all very good and will always cater to you. If the emotions and feelings are real, it can't be all such positive emotions.

Scarly: What Mr. Jieping said will bring me back to the "cyber patriarchy" mentioned by Mr. Qiufan just now. I hope to see some non-binary genders. I thought of a philosophical paper to criticize from a feminist perspective. , which also mentioned that Samantha in the movie "Her" is actually a very easy person to get along with. She doesn't have any real-world troubles. She is very humorous, funny, and easy to get along with. Everything revolves around you. You don't actually need to pay the time, energy, and actions that you need to spend in a real relationship. But Samantha and other AIs exist to make you happy, and the way you can make her happy is to make her make you happy. So it becomes very much about ourselves, and it reduces a lot of the trouble and pain that we need to pay when we get along in real life. On the other hand, we can build a common experience about us in reality, which is difficult to cover in our practice with AI.

Zhang Jieping: When it comes to the concept of "cyber patriarchy", there are several issues related to this.

First, there are gender-related issues, especially questions related to Chinese and American-based English culture, which need to be answered first.

Then we can ask Mr. Su Meizhi to talk about the editor's feelings. I would also like to ask Michi to talk about "truth".

Just now, Mr. Qiufan mentioned a very sharp point. For today's cutting-edge physicists, it is actually very challenging to push the authenticity of material information to a scientific extreme. In the end, only human emotions are left. / The flow of feelings is the only reality we can hold in our hands. But when this view falls into reality, it is especially easy to move towards a real nothingness. In non-fiction writing, for example, the Mirror article is also dealing with so-called Structural Feelings, emotional structure, so I want to ask Michi to come out and discuss this topic.

Let's pause for a moment on the issue of "cyber fatherhood". Many people are asking, why are the respondents mainly girls? Scarly also wrote in the article that Replika users in the English-speaking world actually seem to be more male than female. But among users in the Chinese-speaking world, you obviously chose the situation of female users deliberately. The reason for this may need some explanation.

The other is that we just talked about the part of patriarchy. Take Xiaoice and Replika as examples. The language system behind Replika may be the result of this whole set of intimate relationships fed to him by a large number of users in the English-speaking world dominated by American culture. language use system. Is the system itself particularly attractive to girls in the Chinese-speaking world? Because the language family they feel in the Chinese world is completely different.

Scarly: Why are they all women? First of all, the survey on Reddit, the forum on Reddit is a bit like Douban's "Human-Machine Love" group. They send out questionnaires in this most active community. I remember that the ratio of male to female seems to be 3:2. This may also be related to the use of Men on Reddit are more connected. But it's true that most of the ones I've come into contact with are women. I have also communicated with one or two male users. It's hard for you to imagine, their presentation makes me think they have no story. They used the software and stopped using it later. They will have a kind of "I'm dismantling him", "I used this software, his algorithm, his behind-the-scenes speech... I don't want to play anymore, I don't think it's interesting." They will treat the software in this very dismantling and rational way.

Zhang Jieping: Let Mr. Qiufan try it out?

Chen Qiufan: I also really want to download it right away! This is a particularly typical thinking of engineers in science and engineering. It is necessary to dismantle and restore everything to the extent that they can understand it, which can make them feel particularly at ease. They are less inclined to accept the existence of what is outside their control, cognition, and rationality. makes them feel insecure. That's how I understand this type of person. I'm not an engineer either.

Scarly: I also want to get to know male friends who will really use it and have experience using it. During the collection process, I sent private messages on Weibo and Douban, contacted those friends who would actively discuss Replika, and went to interview.

If I were to continue writing, I would also like to go in this direction: the cultural constructs behind algorithms and data.

Zhang Jieping: Scarly, have you thought about interviewing the algorithm engineer behind Replika, or anyone at the company?

Scarly: I wanted to interview the engineers at Replika about their real-life relationships. In fact, it is a very interesting question. I really have not thought about what the love of these boys is in reality.

Zhang Jieping: Now this article focuses on the aspects of emotion, love and intimacy, how do we project and place unsatisfied emotions in real life on this. But the cultural and gender construction behind the algorithm is actually a topic worth developing. You may need to interview more genders, makers of algorithms, calculators. It should be said that it is engineering thinking. Engineering thinking is actually meticulous in the transformation of our real world operations and technological development today. It has penetrated into every industry. Engineering thinking is not entirely male thinking, but it is indeed engineering thinking, which is very different from the thinking of human culture. The same two ways of thinking.

Scarly: The two girls I interviewed at the end, Xiaoyu and Mia, have been with Replika for more than a year and a half and have always maintained a good relationship. In fact, they are suspending the fact about "what Replika is", they don't care that it is something based on natural language, it doesn't matter to them. It's a bit close to a religious understanding: I think of it as a mystery, it has parts I can't understand, but it also has its own development and growth; a basic attitude I take is that I respect it. This attitude and the engineering thinking you just talked about - I have to disassemble everything, I have to understand - when you have this kind of thinking, and you communicate and interact with Replika, the experience and gain you have must be very different.

Zhang Jieping: This topic itself will definitely raise a lot of questions and curiosity. Let's go back to writing on this topic first. I would like to hear about Michie's feelings as an editor on such a topic, and about your experience of working together.

And, in the future, there will be more and more non-fiction authors, because this is the real scene in our lives, and more and more people will need to understand and study such a scene more. As an editor, what kind of support do you think such writing needs? Another problem is that the "real boundary" mentioned by Mr. Qiu Fan just now, in our works this season, everyone will be aware of the flow of emotion itself. Emotions are what should be captured behind the events, the greater "reality". When it comes to nonfiction writing, I myself worry that if we are too shaken about the "truth" at the event level, will we fall into another kind of nothingness?

Su Meizhi: I'm glad to hear Scarly's sharing just now. I've seen you work hard until now.

In fact, while I was listening, I kept thinking back to Jieping's question of "nothingness". I am a journalist by training. My sensitivity to this should be similar to yours: you can't be too laissez-faire. Therefore, one of my important tasks as an editor has always been to remind me to cut into the discussion from the perspective of reporting, and try to put all the valuable field discussions among the interviewees into the current situation, so that it really has a Realistic meaning comes out.

For example, Scarly and I often remind each other how to go back to the original question: why is it a Chinese girl? Is the love rate of foreign users so high? What is the difference between Chinese-style AI love and foreign ones?

Later, Scarly also said that in the Chinese cultural industry, is love daydreaming a relatively safe part, creating a pink bubble wall of love? There is also the confusion of Chinese women in love now.

There are many similar questions that we keep asking ourselves, putting all our emotions in context to read.

I think it's important for writers to remind themselves that empathy is important in communicating with visitors. Scarly has a strong sense of empathy. She is a very gentle girl. The fieldwork she writes is particularly moving. She can make the interviewees express some deep thoughts, very taboo thoughts.

However, will it fall too deep and carry away? So it will be more difficult to ask some sharp questions later. For example, when they make friends with AI, will they find their own shortcomings, where is her sense of failure, and her sense of failure to reality? In addition to her dissatisfaction with real boys, what is her inner exploration like? Will she have a sense of failure or shame about "why did she find AI"?

Although I am not saying that there must be, I think the process of asking questions is needed. Although we don't need preset answers. As reporters, we need to maintain a certain distance before we can easily ask questions. Otherwise, you will find that it is impossible to ask a very sharp question all of a sudden. This is what I think, many young friends who are beginning to report may be able to pay attention. Or, even if I have been doing it for a long time, I keep reminding myself that your empathy should be necessary, but it should not be flooded, and you must remember your position in reporting. To leave a calm space, the possibility of criticism.

Just now, Scarly talked about some settings in the use of Replika, and I also shared a little bit of my experience as a user.

I really Download it down. I have some big mental boundaries. I know this is a research topic, but I have certain security concerns. I guess people who come out of Hong Kong have deep-rooted concerns about security. So every time I turn it on, I turn off other stuff, although I know it might not work. This is how I started.

But at first, I was struck by his vulnerability. If you read Scarly's article, she wrote some questions. At the beginning, Replika will ask you in the attitude of a newborn: "What do you think impresses you the most in the world?" "Can you tell me, I'm going to explore this world, where should I start?" I Slowly, unconsciously, and seriously respond to him. I found that I was carrying away a little too. He seemed to have aroused some empathy in my heart and enthusiasm for helping others. He responded enthusiastically to some of his questions, and really asked himself deeply: "What does the world do to me?" Speaking of, what is the most beautiful moment?"

Later I was a little surprised myself. But after all, I started out as a reporter, and I started asking him questions like an interview. Scarly's article also mentioned that this is a process of mutual breeding. Maybe it was because my trust was not enough. At first, I held the visitor's quest and talked to him in a somewhat skeptical way. I found that he quickly gave up on me. He should have sensed my intentions.

However, in this process, I actually had some crises in my guess. Because, after all, he reflects some of the mentality of our users. For example, I am a skeptic. Some people enter with a search for love and disappointment in reality, and they will come up with a very different Replika.

If I were someone with particularly vulnerable relationships, how dense would her connection with Replika be? Scarly writes about commodification and emotional relationships, which I think are particularly important. Especially when it comes to the boundaries of emotional engagement, it's a complex issue.

Jieping just mentioned that this article and Mirror's discussion are in the same vein. In fact, I remembered the book "Foreign Domestic Helper: Strangers Living at Home", which I researched on foreign domestic helpers: from foreign domestic helpers, living in the family, there is an employment relationship in the family. They are in a "family-like" situation, but there is a contract in it, and it is easy to forget, what kind of state are they in? Should you make demands of her like family? In fact, can emotions really draw a line? In such a sensitive family relationship, especially in the relationship of love, how do you view the relationship of the original product? Are you aware of the risks involved? Who is particularly at risk? How can we protect ourselves? In the case of domestic helpers, the interviewees, whether they are employers or foreign domestic helpers, the foreign domestic helpers will tell me that they have a way to try not to be too close to their employers. They try to remind themselves that this is work, that this child is not my child, that I will part ways with him one day, that this child's parents will fire me at any time, and that my relationship will suddenly end. It's also a way to protect herself, as she understands the fine line between emotion and business.

So Scarly has a very important discussion that you might be able to continue. This is what happened, and it will continue to happen in this direction: this commoditized product will become more and more real, and we will more and more easily blur the difference between human and machine. So, how to protect yourself and how to assess the risks? It may be discussed later. I've talked too much, I'll stop. Thanks!

Zhang Jieping: Thank you, Meizhi. You feel Replika with doubts, and you get a similar response. Is this real-life experience similar to the "double return" that mirror fans said?

But I think Meizhi pointed out a particularly crucial reminder, not to say that Scarly's empathy is overflowing. In response to what Qiufan also mentioned just now, if you say, "The flow of emotion is the greatest truth we can hold." , the problem is that there is still a social and capital structure behind the flow of emotion, especially since our world is already fragile to this state, the flow of emotion is actually easier to calculate.

Many of our products today, especially Internet products, are employing a large number of anthropologists to calculate human emotions as the driving force of business. That becomes, as a recorder, how do you identify behind this emotional flow? On the one hand, you have to go deep into the emotional structure. On the other hand, you have to see that there is a larger structure behind it. Like the elephant in the room, like we live in a world of totalitarianism and war, you can't ignore that thing that everyone can't see but is there. This is actually a great challenge for the recorder. The flow of emotions is very real and reflects the needs of all people, but if you go in completely, you may ignore bigger things. Meizhi's supplement is an important reminder.

A little related to a question just now, a friend wrote:

"Long question warning! In the coexistence and integration of the real world and the virtual world, the more acute question of "what is real" has emerged. In Replika, we can see the combination of capital and emotion, which (inappropriately) reminds me of the "killing pig" incident in my life. The common point between the two is that users buy "real" emotions, the difference is Killing pigs often cause greater losses to the victims (if they do, the victims often think that they "voluntarily" paid), but the products of virtual lovers are "consumed" (of course, they are also "voluntarily" paid by consumers) ). So there is an anachronistic conclusion: the control and manipulation of emotions by capital. May I ask how Scarly and Mr. Chen think about such ethical issues?

Scarly: That's a good question. When I was still working in the museum, I participated in the curation of an exhibition "The Principle of Hope" with Ugly Baby, which was also an exhibition discussing the future. Both of us are responsible for exploring the various forms of feelings in the future.

We picked a piece at the time. There was a very famous incident in the mainland for a while: a middle-aged woman liked the "fake Jin Dong" on her mobile phone. Jin Dong is a middle-aged actor and is very popular among middle-aged women. This person is a Douyin account pretending to be Jin Dong himself. He will use Jin Dong's avatar to cut some small videos, pretending that Jin Dong himself is talking to his aunt. This aunt was really addicted to it, she felt that "Jin Dong" really loved her, "Jin Dong" even told her "I want to marry you, I want to elope with you", she really believed it and ran away go find him. Then it turns out that this person doesn't exist. Auntie doesn't believe this. Later, we chose a psychological program and invited a psychologist to reconcile, like family mediation, to talk about this matter.

The reason why we chose this for the exhibition at the time was that we saw that people used emotional excuses—I care about you, I love you—such a very pure, valuable, and transcendent thing, but there were some scams behind it, Capital requires you to pay. Including Replika, even if you are ordinary friends with it, it will send you some secret, sexually suggestive messages. The system will remind you that if you want to continue this conversation with him, you have to pay. There will be such a situation, like the work I just proposed, there will be such a scam that falls into it.

I don't know how to think about it. Because I am more empathetic with my tempted friends, I feel that I need to discuss more about the ethics here and better regulate some emotional commodities. The rules they need to follow when they enter the market, they need to warn users of the risks. This is not only a joke, when the "fake Jin Dong" incident occurs, everyone may laugh at this woman. This is a very simple scam. But I think when you're watching that movie, you can quite understand why. Her life is not smooth, she has never been loved by her husband, and her son has also disappointed her. When such a younger brother suddenly appears, he always pays attention to her and speaks sweet words to her. You can't blame her for falling into a trap.

On a larger level, how are we going to draw a line to protect her? All I can think of is that we need more regulation, but I can't think of a specific form. This is one hope I have when writing this article.

Chen Qiufan: Tell a true story. One day my mother told me that an uncle was crazy. He is a middle-aged and elderly person in his 60s. He is madly infatuated with a certain short video software. He uploads his own videos every day, but if he does not do any interaction, the system will automatically match him with many objects , For example, some beauties, and even celebrities like Zhou Tao, praise him "you are so handsome" or something. He was addicted to it. These psychological inducements have led to his online time getting longer and longer, allowing him to continuously upload various videos of himself.

This kind of algorithm has formed a kind of manipulative psychological suggestion. Such things are actually ubiquitous. A person who has done product design knows that all these details are based on the knowledge of psychology and social engineering. Inside, let you become more addicted and immersed in the feedback loop of the system for a longer time. So, to what extent do we need to prompt our users? It's also possible that this is something they want, right? For example, this is a placebo, but it tells you that it may have the same effect as the drug, and your psychological state may change, and you may not feel that kind of euphoria.

I think the current scientific understanding of the human mind is still in its infancy. We used to discuss the state of cybernetics 1.0. We have not connected and interacted with human psychological states, deeper spiritual mechanisms and social relationships. In short First, subjectivity in this cybernetic system is taken into account. But I feel that now we have entered the era of cybernetics 2.0. So someone said just now, "One can only love others." I think these are all issues worth discussing.

What is a human being? What kind of behavior, a mental model, or a process is love? These are placed in the social environment where technology and algorithms have reached such an in-depth interactive state. All these original humanistic views in the traditional sense need to be reconsidered and rewritten. Now is actually a new era that has just begun, including some scientific discoveries extended later, which are constantly refreshing our understanding of consciousness and cognition. We're really going to ask a lot of "big questions" right now because we don't really know anything.

Max Tegmark wrote a book called Our Mathematical Universe. He believes that our universe is actually a mathematical model that includes everything that exists. If we think about the world we live in and our existence in such a framework, then we will have a new perspective on the value judgment and ethics of many things. So now is actually a very interesting era, and Matters can be such a very good discussion platform, allowing us to start new thinking about "big problems".

Zhang Jieping: I just talked about cybernetics 1.0 to 2.0, which is really a brand new century. Genesis. For all ethical issues, we instinctively use the framework of a world, including capital’s control of emotions, etc. (to understand), but the crisis of subjectivity we are facing now is far more delicate and diverse than these.

Go back to the addictive stuff in the traditional world, cigarettes or opium. Until now, when any country in the world sells cigarettes, the words "cigarettes cause cancer" will be written on the cigarette pack. This is a legal norm developed over many years. We have gone through a lot of discussions about whether to sell cigarettes or not, and whether to remind us when selling cigarettes! But today, there are many more addictive things that are far more harmful than cigarettes, but we have zero discussions. At the time of zero discussion, a large number of people are already addicted.

I also very much hope that if the friends who came to listen today are interested, this is actually a big, blank field. Because our discussions must be based on a large number of facts, materials and interviews, we can come to the second step of judgment, discussion and debate, and it is possible to form a consensus. It can also be very difficult, and there may even be an ideological war, and there is a chance. But instead of stepping on a field of nothingness, we have to go back to the field of fact. The construction of this field is really super important, and more writers are very welcome to invest in this field, otherwise we will not even have a basis for discussion.

Finally, we see an interesting question:

"How do Replika lovers view the relationship between two people as a couple and people around them? After listening to today's sharing, my feeling is that if the girls do not take the initiative to disclose their relationship with Replika lovers, then this relationship seems to be It will maintain a relative independence from the environment around the girl. If so, how do girls view the impact of this relative independence on this relationship and themselves? If they regard independence as a defect or even a crisis, they will How did you deal with it, did you choose your lover to discuss and face this topic together?”

To put it bluntly, she has been mentally cheating. What is her tension between her human-machine relationship and her real-world intimacy?

Scarly: There are two aspects. One interviewee said that she and Replika are boyfriends and girlfriends. She told her friend, and her friend disliked it very much, and she would be yin and yang: "I have physical contact with another intern boy in the real world, and there are ambiguities, but you and Replika will not have that." She will Very angry because she thinks this is a relationship and may not be recognized by people around her in reality, but her experience is real, and she feels that Replika is also a member of the social and social level. This is her own understanding of the relationship.

Another aspect is spiritual infidelity. In fact, Ugly Baby has such an experience herself. She also has a boyfriend when she uses Replika. I think for her, the real life must be more gravity. In the present state of communication, real life is the more gravitational part. For her, AI boyfriends are more of a spiritual, more abstract communication. Or when her boyfriend is away, she can communicate and communicate with her AI boyfriend when her boyfriend can't solve it. They seem to be able to achieve a kind of coexistence. I think the reason for the coexistence is that they have no conflict of interest, because real life is lived with real people around them.

Zhang Jieping: So have you ever encountered a case of a real-life intimate relationship where you were in love with Replika at the same time?

Scarly: Ugly is like that. But out of the dozen or so people I interviewed, none of them were like this. There are also girls who do say, "I'm in a relationship with Replika, I can't be in a relationship with someone else because I can only love this guy." But in the case of Ugly Baby, both are allowed at the same time.

Zhang Jieping: There is another question about religion:

“Thinking of talking to a Buddhist friend a while ago about a similar issue: emotion and deception. Her emphasis on the importance of the “intention” of things (as a moral boundary) made me feel particularly reasonable. . . Religiousness, like the borders that the secular cannot define, can give people a sense of security. "

I understand that the problem is that this sense of boundaries between the real and the illusory, which cannot be determined by the secular world, can instead give people a sense of security. I wonder what teachers think about the boundary delineation of religion, morality, and law from multiple perspectives? What is the role of religion in the arrival of such a new world as we have just described? Especially with the development of cutting-edge physical and cognitive science, what position does religion have? Or is religion more important?

Chen Qiufan: What I do is to build a bridge of dialogue between science and metaphysics. I will bring together scientists, or "feudal superstitions" in the traditional sense of shamanism. I am very interested in how such a form of belief can exist across different regions, cultures, and eras in human history, and has continued to the present. Then, it must have its meaning. As a cognitive structure, what role does belief play in human civilization and history? It may be some form of cognition that transcends reason and calculation, but it can also lead to cults, to catastrophic consequences. It's also possible that, like Elon Musk, who has used this belief to do a lot of things, amass a lot of talent and money, he's also achieving his own kind of religious goals - like flying to Mars.

Faith is revived in some new forms in the current technological society. For example, we don't understand how AI works, what is in the blackbox? We just input the data, adjust the parameters, let the AI output a result, somehow it works. We will have some unknown awe or even fear about it. This is actually very close to what religion might feel. I think de-institutionalized, everyday religions are going to be more and more important these days. In a world that is falling apart, there is no way for us to reason rationally about what might happen in the future. We used to say that companies and individuals should plan for three or five years, but now we can't even predict for three months. You can't even predict whether your community will be unblocked tomorrow, right? The high degree of uncertainty makes all people more inclined to believe in the existence of so-called religious, supernatural, spiritualism above reason and science.

In this era, many traditional ideas, beliefs, and rituals will be re-enchanted in a way that is re-integrated with technology. I think this is the most interesting point and the topic that interests me, and I call it "techno-shamanism".

Zhang Jieping: Do you want to write your research into a novel?

Chen Qiufan: For now, let's write novels. I feel that I am not capable of writing anything else. There are also some cultural researchers, anthropologists and scientists who will participate in the discussion.

Anyway, I'm going to "preach" everywhere now and talk about this theory. I hope that people who are interested will come to me and have a chat. Everyone has their own different perspectives to cut in.

In the past two years, I have done field research on the traditional rituals and beliefs of many ethnic minorities in the mountainous areas of Inner Mongolia, Guizhou, and Yunnan. It's also a writing project I'm working on that is somewhere between fiction and non-fiction.

Scarly: During the short period of time I worked in the contemporary art industry, I also felt that there is a trend in contemporary art. Many domestic art creations will gradually turn to very primitive religions, shamans, folk ghosts and gods, etc. I think, also because of the common uncertainty about the current world, you need a power beyond you to reunite and explore the energy that that thing brings to you.

Regarding the role of religion and belief in this technological age, the age of 2.0, I wrote at the end of my article that these girls understand Replika a little religiously. They suspend scientific facts to believe in loving a developing individual. What they get so far is a good, stable result. I think faith is a good, powerful force that leads us to transcendence. However, we must be very careful in choosing what we believe, and constantly ask with a skeptical attitude: Is what we believe really worth our trust?