Google Gemini and the problem of social discrimination in artificial intelligence
Share
Artificial intelligence (AI) is the great technological revolution of the moment andcompanies are investing to integrate the different advantages it offers. A clear example is Google with the launch of Gemini the latest version of its AI-based generative model. Gemini is an evolution of its previous model, Google Bard, and brings an outstanding feature: the ability to generate images from textual descriptions.
With this new feature, a high-quality AI-generated image is generated from a description in just a few seconds. This is a major milestone in society because it brings this innovative technology closer to the people, while providing new tools to the entire sector that is dedicated to it professionally.
What are the problems with Google Gemini?
The launch of Google Gemini was widely welcomed by the public, subjecting the model to a wide range of descriptions. However, after a few days of testing by users, controversy arose.
Social networks were filled with numerous messages highlighting an unexpected feature of Google Gemini: discrimination and diversity. When developing this AI image generator, it was trained with a special emphasis on racial and gender heterogeneity in photographs involving human subjects, in order to provide more diverse and inclusive images.
Criticism pointed to a lack of rigor in creating images that accurately reflected the reality of historical events and situations. Several users expressed their dissatisfaction with Google Gemini when they noticed that a request to generate images of Vikings (people originating from Scandinavian peoples), yielded a considerable number of results with racialized individuals of black descent. Similar cases were reported for searches related to Nazi soldiers or the founding fathers of the United States, along with a long etcetera.
What problems has Google Gemini caused?
In addition to the great media hype, the controversy generated by Gemini has had a considerable economic impact for Google. After the news broke on February 26, 2024, shares of Alphabet, Google’s parent company, fell by 4.5% and Forbes estimates a loss of value for Google. Forbes estimates a loss in market value of $90 billion.
Les vikings selon Google Gemini ! Le wokisme est une maladie mentale. pic.twitter.com/dxCsZXoStx— No surrender (@HubertMillo) February 28, 2024
Faced with this situation, Google acted quickly by pausing the generation of images in its tool. Its CEO, Sundar Pichai, openly criticized the lack of historical accuracy of this model produced by the company, followed by a publication on X (former Twitter), apologizing, as well as another written on the company’s blog by Prabhakar Raghavan, senior vice president, explaining what happened.
This problem with Google Gemini is not the first one the company has had. In this line, a few years ago when searching for images of gorillas in Google Photos, searches for black people appeared. After an apology from the company, they proceeded to remove the ape category from the training of Google Photos AI models.
Nuevamente Google perdió dinero por sus IAs. Las imágenes erróneas que creó #Gemini sobre soldados alemanes nazis de la II Guerra Mundial con piel negra o rasgos asiáticos hizo que las acciones de la empresa cayeran un 4.5%, perdiendo 90,000 millones de dólares. pic.twitter.com/fERrN38nvW— Javier Matuk (@jmatuk) February 28, 2024
Other problems with artificial intelligence (AI).
Both cases represent opposite effects of the same problem: the existence of biases in AI. This is not a dilemma unique to a company like Google, but a more general problem in the field of artificial intelligence that is exacerbated by new language models and generative AI. It is clear the negative effect this problem can have on society, not only in terms of trust and reliability of such systems, but also on the implementation of this type of technology in certain areas.
For example, the U.S. legal system uses an AI-based tool called COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) to estimate the risk of recidivism in the prison population, serving as an indicator for judges to make certain decisions regarding prisoners. This tool has been shown to produce biased predictions based on race, tending to rate black or Latino people as higher risk than white people.
Also noteworthy is the case of Amazon, which in 2018 had to scrap an AI model used to classify resumes, as the system was shown to discriminate against women for jobs with a technological profile.
Another well-known example is Apple’s credit cards, where an automatic system systematically denied granting a bank credit to a woman, while granting it to her husband, even when the finances within the couple were shared. Despite Apple’s claim that its algorithm does not consider gender as a parameter, the results suggest that, in some way, the algorithm incorporates an implicit bias in relation to gender.
Another example is the well-known Deep Fakes, those images that a priori seem real, but are created by AI. Here there are tools such as VerifAI capable of detecting whether they have been manipulated and with what programs.
Artificial intelligence is undoubtedly making great strides, although from time to time it is necessary to stop and see if everything is developing correctly.
The presence of biases in AI
We know that AI-based learning algorithms, and especially in the case of generative AI and foundational models, often need large volumes of information to be trained.
As a general rule, the larger the volume of data available to train these models, the higher their accuracy. However, thisHowever, this implies that the algorithms tend to pay little attention to those population groups that are less well represented in the data, such as minorities.such as social minorities, among others. This often leads to a higher error rate in the predictions for these population groups.
The problem is exacerbated as the volume of data increases, as in the case of generative AI training. In addition, the data used to train these AI algorithms reflect existing biases in society. When used in training AI models, these biases are replicated and even amplified.
The widespread adoption of generative AI in various areas of society makes it essential to develop techniques to mitigate problems related to algorithmic fairness.
A problem not so simple to solve
The analysis and mitigation of the effects that these biases can cause are not so obvious and straightforward. The problem of diversity and discrimination in AI algorithms has been studied for years. However, there is a trade-off between the accuracy or usefulness of models and their diversity or non-discrimination.
Thus, algorithms that take this diversity into account or try to reduce certain biases may see their accuracy reduced compared to algorithms that do not take this aspect into account. In certain applications, this reduction in performance is justified by the ethical and social impact. However, as in the case of Google Gemini, these inaccuracies may not be acceptable in certain contexts.
Often, generative AI models are designed to perform a large number of tasks and to be used in many contexts at the same time, as is the case with Chat-GPT or Google Gemini. This makes it very difficult to foresee and manage all scenarios where an algorithm is required to generate more diverse and inclusive responses or where greater rigor and precision is required.
Google Gemini’s algorithmic solution to improve racial discrimination in generated images is beneficial in certain contexts, but has a very negative impact in cases where, for example, historical rigor is required. This makes it clear that a one-size-fits-all solution is not recommended.
The unstoppable advance of AI
As a lesson learned from this situation, we point out the need to carry out complete analyses in the design phase of this type of systems, evaluating their intended uses, as well as potential abuses and malicious uses. This must be done not only from a purely technical, but also from a social perspective. This prior step is essential to assess the impact and ethical, social and security risks of these AI algorithms.
With the unstoppable advance of AI in many areas of everyday life, looking the other way is not an option, as these aspects become highly relevant and can undermine confidence in this type of algorithms if they are not designed in a responsible way, which can have a negative impact for many companies.
This post has been written in collaboration with David Solans Noguero,Scientific Researcher at Telefónica Research.“
Senior Research Scientist en Telefónica Research. Doctor por la Universidad Carlos III de Madrid en IA, donde recibió el premio extraordinario de doctorado por su tesis doctoral. Cuenta con experiencia profesional trabajando como investigador en el Imperial College de Londres, donde se ha especializado en temas relacionados con la fiabilidad y seguridad de los algoritmos de IA. Es autor y/o coautor de numerosos artículos y contribuciones científicas de diversa índole y tiene experiencia en participación y liderazgo de proyectos de investigación.
