Frenemies: Artificial Intelligence is already influencing our behavior. How do we keep it in check?

The AI Revolution is happening slower than we expected

Artificial Intelligence and Generative Artificial Intelligence have been hot topics in recent years, especially since OpenAI launched ChatGPT, in November 2022. Companies like Microsoft, Apple, Anthropic, and Google are also competing to develop software based on these technologies. This sparked widespread interest, as these tools became accessible to the general public.

As technology evolved and became versatile enough, AI assistants gained advanced capabilities. From conversational skills to image generation and the ability to provide summarized information directly in search engine results. This has set in motion a “technological snowball” that cannot be stopped and that will revolutionize our lives over the coming decades, profoundly changing how we inform ourselves, socialize, and make a living.

While most associate the idea of revolutionary technology with overnight change, the AI Revolution is a gradual process. Some experts compare AI development impact on the world to the Industrial Revolution’s transformation of societies in the 18th and 19th centuries. Since the process is relatively slow, we might not fully realize that what we are experiencing now represents an unprecedented turning point in human history.

Jennifer Victoria Scurrell is an expert in Human-AI Interaction at the Center for Security Studies (CSS) at ETH Zurich, Switzerland, and Member of the OpenAI Red Teaming Network as an Independent Contractor. The latter is a network of external experts specializing in security, ethics, and risk assessments related to Artificial Intelligence, invited to test OpenAI’s systems to identify vulnerabilities, risks, or potential harmful uses of these technologies.

When she first heard about AI, Scurrell was expecting a major change, like a sci-fi movie makeover: “I was always interested in technology, I was always very positive about AI, about having machines in the future that are like big motherships, and we are traveling the galaxy. So, I had very romanticized the AI, like I also romanticized cyborgs”, she explains in an interview for Panorama. “I still believe that it’s an interesting and a good thing for humanity to let machines support us. AI is actually here to not only support humans, but make them better”. Scurrell adds.

Lauren Treiman is a PhD student at Washington University, where she conducted, along with two other colleagues, a study about how AI changes human behavior in decision-making. Even before starting the research, she percieved AI technology as a very powerful tool.

“If you think about it, it is all around us. […] It’s even being used in really important contexts, people are including it in college admissions, in the judicial system and so on. We’re constantly using this, so it’s important to understand what’s happening when we’re using this”, she says in an interview for Panorama.

In the United States, 5% of the companies use AI to produce goods and services, notes a report from 2024 from United States Census Bureau. In EU, 8% of the companies used AI in their business processes in 2023. Romania is on the last place, as this Panorama infographic shows.

AI, the friend we didn’t know we needed

Artificial Intelligence has a major impact not only in business, but also at a human level, where it can become a factor that facilitates socializing.

Loneliness among the elderly is a global problem, and companies have translated AI technology into robots to keep them company. Japan, already with more than two decades of experience in the field, uses robots such as Pepper, AIBO or PARO in care homes for the elderly and hospitals. They are meant to “entertain, start conversations and create emotional bonds” with their owners, notes an article in Spanish daily El Pais.

Based on the same idea, the Israeli company Intuition Robotics has also developed ElliQ, an “AI care companion robot designed to empower independence and promote healthy living,” says the firm’s website. Earlier this year, the third generation of the assistant was launched, as it “proven to positively impact seniors’ health, social connectedness, and independence,” a company press release said.

AI systems are also used as assistants for people with post-traumatic stress disorder (PTSD), thanks to the homogenous and “human-like” interaction the technology offers. Jennifer Victoria Scurrell noticed in research that war veterans who come back from the battlefield and are suffering from PTSD feel it is easier to talk about their experiences with an AI assistant, “because they’re so ashamed of what they did during war”.

The impact of biases in algorithm decisions

Beyond being our conversation partners, AI assistants also generate other important behavioral changes. The research run by Treiman and her team shows that people tend to behave differently when interacting with AI technology. Especially since many of the scenarios for training the technology involve the fact that humans are helping to improve decisions made by AI.

“We were curious to see if people know that they’re training AI, will they bias it. What we showed is that, when playing an economics bargaining game, the Ultimatum game, people over-exagerate their willingness to be fair. In fact, become more punitive to reject lower offers, to bias the AI and teach the AI such that being unfair is not OK. So, people are infiltrating that bias there”, the researcher explains for Panorama.

Although at first glance the results are promising, Treiman underlines that things are not as simple if we think of real-life scenarios.

“In the real world, there’s a lot of time not this one definition, what I consider to be fair in one thing, you might disagree. So now this becomes a little problematic, if I am changing my behavior and over-exagerate my bias into the AI algorithm. It might not reach its goal of deciding what is fair based on what overall population thinks, because now I am exagerating my behavior and I am not providing a representative bullet point”, Lauren Treiman details.

This is how AI systems end up perpetuating human stereotypes or behaviors when receiving prompts from users. Consequently, their responses indirectly reinforce these behaviors among those using AI assistants. That might not seem so bad if we think about positive behaviors. However, one of the attempts from 2016 to make AI available to the general public reveals that the situation requires a cautious approach.

Eight years ago, Microsoft launched Tay, an AI bot on X (then Twitter). But, the company had to pull it quickly, because users had trained it to tweet and make discriminatory remarks, Nazi, and sexual remarks. If we extrapolate this scenario to today, where 200 million people use ChatGPT on a weekly basis, according to an OpenAI announcement cited by Reuters, the situation becomes delicate.

Useful tools, not sources of total trust

Scurrell says that the AI technology also brings the dangers about misinformation, with the influence of political opinions being swayed by deep fake material, as Panorama also wrote.

“We really need to train from preschool how to deal with technology and how to critically think, in general”, she adds.

Lauren Treiman also thinks AI could be a problem if there are bad intentions: “You might get some poople out there who just might say «let’s mess with a bunch of people, let’s mess with this AI algorithm»”, and that could have negative consequences. Especially because, Treiman says, while it may seem like AI is all-knowing and highly accurate, the reality is different.

“These AI systems, like ChatGPT and other Large Language Models (LLMs) are really good at what they’re doing, but they don’t know what they are doing. They’re really good at writing or looking up stuff and they are getting better at their accuracy – everything sounds legit. But if you tell people «hey, look how smart this AI model is, it produces really, really sophisticated results», you just think «wow, we can have a general overall reliance on this». Well, that’s not really what the case is”, Treiman says.

In reality, ChatGPT is good at predicting the next letter or the next word, but it doesn’t understand the meaning of the sentence, because it doesn’t have logic. The algorithm chooses words based on previous interactions without understanding the context, she explains.

That’s why she thinks AI systems can be used more as tools, rather than pillars on which we can lean completely.

How do companies make sure that the algorithm is fair

Another side of the problem is that the software industry is dominated by certain perspectives and ethnic and social categories, and that influences not only the data that the products are trained with (usually data from the Internet, a massively biased medium), but also the way they are programmed. “Still so many engineers are young, white, male, and they don’t ask the right research questions. We need more plural thinking into the companies, into the engineering”, Scurrel explains.

To avoid accidental biases that can negatively influence human behaviors, many AI companies have formed Red Teams, teams with various backgrounds, nationalities, cultures, and preocupations. Their role is to interact with LLM models, test and make “fine-tuning adjustments” to algorithms, says Jennifer Victoria Scurrell, an independent contractor with the OpenAI Red Teaming Network.

“The model works, but we basically tell where to put the guard rails. And where does the model misbehave, not only hallucinations (note – software errors that generate nonsensical or wrong information), also hate speech, persuasion, slur words, but also problems with copyright”, the researcher explains.

AI companies should think about what effects the systems they release have on human behavior, Lauren Treiman believes, because a software system doesn’t always do what its creators think, and humans can influence it.

“If AI developers think that their data is unbiased, I encourage them to think how human behavior is changing the AI, from what it’s wanting to do, to what they’re doing. Because if they don’t, there’s gonna be selling this AI like this is an unbiased recommendation system, when in fact that’s not what it is. It’s better to acknowledge the limitations of the model instead of just not think about if it is there or not”, Treiman adds.

Jennifer Victoria Scurrell also believes that the way AI models influence people also depends on the developer’s intent. There are malicious actors who could bypass the security measures built into AI models and negatively influence them in such ways to affect people.

“Technology is always a mirror of society. We’ve seen it with social media, where there were good things, but they were also a lot of very bad things, because we kind of missed it to regulate it. And it’s good that we talk about regulation that makes sense. I don’t want to not have my freedom, but I do understand that there must be some regulations for the minimum”, Scurrell says.

The researcher adds that the policies should be made in such a way to not complicate things excessively and to not hinder inovation in the field.

AI regulation becomes vital for a safe future of humanity

A growing number of experts are concerned about the impact that Artificial Intelligence could have on the future of humanity if AI systems evolve too fast to be controlled. It’s what experts call the technological singularity, a threshold that, once crossed, will allow Artificial Intelligence to surpass the power of human cognition and improve itself. So, to become autonomous. This point is also called Artificial Superintelligence (ASI or Artificial General Intelligence – AGI).

Incidentally, Ilya Sutskever, one of the co-founders of OpenAI, left the company and, with two partners, set up Safe Superintelligence (SSI), a safe ASI development lab. “Superintelligence is within reach”, reads the first sentence on their one-page “letter” signed by the founders on their website.

So far, no AI developer has managed to officially create the technological singularity, although OpenAI’s missions include “ensuring that General Artificial Intelligence – AI systems that are smarter than humans – are safe for humanity”.

AI technology will have an had to imagine impact on humanity, and most people underestimate the positive effects of this technology, Dario Amodei, CEO of Anthropic, the company that developed the AI model Claude, writes on his blog. We might reach ASI “as early as 2026”, he emphasizes, and Artificial Superintelligence will allow us to make previously unimagined advancements.

“I’m actually open to the (perhaps absurd-sounding) idea that we could get 1000 years of progress in 5-10 years, but very skeptical that we can get 100 years in 1 year”, the CEO of Anthropic explains, refering to the power that such a technological discovery could bring.

It seems that the future Amodei is talking about is already here. At the end of last year, the media reported that a number of OpenAI researchers had warned the company’s board about a discovery that could become a danger to humanity.

In fact, current and former employees of the company published an open letter in July this year, warning about the danger of Artificial Intelligence on the future of mankind and the need for strict government regulations to avoid negative scenarios.

“AI companies have strong financial incentives to avoid effective oversight, and we do not believe bespoke structures of corporate governance are sufficient to change this”, say the signatories of the letter.

The European Union’s Artificial Intelligence Act, adopted this year, responds to this problem by creating a legal framework to regulate the use of AI technologies in a way that ensures safety and respect for fundamental rights.

How the human-AI relationship will evolve

Although the topic of technological singularity is stirring intense concerns and discussions, and many organizations have been created to protect us from the risks of such processing power, Treiman and Scurrell encourage a balanced attitude.

“In the next years there will be personalized AI. They’re becoming more human-like – they’re becoming your buddies, they will become personalized on your needs and they’ll be taking away the boring work from you. And this is really amazing”, says Jennifer Scurrell about the future of the human-AI relationship.

Lauren Treiman also thinks that “AI has a lot of benefits for us and society. It is making so many great advancements. AI is the future, so we have to embrace AI and it just comes down to learning how to properly use it. […] There’s a lot of AI ethics, and fairness, and AI transparency going on, which I think is super helpful, because just knowing how to use it it’s just going to help. There are some potential shortcomings, but I think, utimately, if used correctly, it will be better”.

Discussions and controversies about Artificial Intelligence remain complex, and opinions differ depending on the side – researchers, AI developers, CEOs or other experts. Although the future cannot be predicted, AI continues to fascinate and provoke fears. It seems to be both a friend and a foe, depending on how we manage it. It remains our responsibility to choose what to do with the new definitions AI will give in the coming decades to the way we communicate, work, and live.