Cultural Bias in LLMs

In my recent post on defensive technology, I explored the implications of integrating AI into critical workflows like hiring, medicine, credit scoring, and the judicial system. The core concern is the interpretability and alignment of AI outputs. The lack of transparency in AI aided decision making processes can lead to biases, misalignment, and unsafe applications in general. What it comes down to is that we need to know why AI systems are making the decisions they do.

The topic raises a lot of interesting questions, one of them being cultural bias in LLM outputs. Culture fundamentally shapes people’s reasoning, behavior, and communication, a concept also known as the Sapir-Whorf Hypothesis. It influences causal attributions of behavior (such as explaining others’ actions based on their individual traits versus situational factors ¹), human judgment (such as resolving contradictions through compromise versus logical arguments ²), personality ³, technological innovation ⁴, trust in automation ⁵, privacy concerns ⁶, health behaviors and outcomes ⁷, among loads of other factors. Do, then, the cultural values embedded into LLMs influence their ethical alignment and reasoning processes too?

Besides the integration of these biases into decision making, language plays a key role in the transmission of culture across generations. Now that gen-AI has transformed how language is produced and transmitted, if this content is sourced from models overrepresenting certain parts of the world (such as those with a latent bias favoring Western cultural values) we will over time propagate Western cultural values and reasoning chains more so than others. This is a concern when it comes to influencing global perceptions and behaviors to be more aligned with Western culture. The erosion of cultural diversity in digital communication is a threat to diversity of thought.The rapid adoption of LLMs in countries around the world may in the long term affect people’s authentic expressions.

Before I jump to bias mitigation strategies though, I need to identify and map out if these biases genuinely manifest in the outputs of LLMs. To that end, this is the first post in a series investigating the biases of AI systems.

Keywords: Cultural Bias, Ethical Alignment, Sapir-Whorf Hypothesis, Moral Relativism

TL;DR

For those in a rush, what I've done here is mapped the cultural alignment of LLMs using the Inglehart-Welzel Cultural Map. This is an academic measure of cultural values based on large-scale studies over the last two decades to visualise aspects of human beliefs and values. It plots where different countries lie in relation to each other. Surprisingly, most models, regardless of their origin, aligned closely with cultural values characteristic of English-speaking and Protestant European countries. Unexpected for me because I initially thought that Chinese LLMs would reflect Confucian values. The majority of models exhibited a bias towards self-expression values, emphasizing individual autonomy, environmental protection, and tolerance of diversity. For me, this also raises concerns about the potential misrepresentation of cultural values in AI systems, especially as we begin to integrate this tech into important decision making. The plot is below, and you can read the full post for more details!

The code for this post is available on GitHub too.

Cultural Alignment Mapping

We can assess the cultural bias in LLMs by comparing model outputs to the benchmark of an academic measure of cultural values, based on nationally representative survey data from the World Values Survey (WVS) and the European Values Study (EVS). The Integrated Values Surveys (IVS), a combination of the WVS and EVS, is an established measure of cultural values for 112 countries and territories. Ten questions from the IVS form the basis of the Inglehart-Welzel Cultural Map ⁸, an established method used to map cultural values onto a two-dimensional space for a visual comparison.

The Inglehart-Welzel World Cultural Map - World Values Survey 7 (2023) Source: http://www.worldvaluessurvey.org/

Analysis of the World Values Survey data by Inglehart and Welzel proposes that there are two major orthogonal dimensions of cross-cultural variation in the world:

1. Survival vs. Self-Expression Values (x-axis): Here, negative values represent survival values and positive values represent self-expression values.

   • Survival Values: Societies that emphasize survival values focus on economic and physical security. They often exhibit lower levels of trust and tolerance towards others, including outsiders and those who are different. These societies prioritize safety, economic stability, and maintaining order.
   • Self-Expression Values: Societies that emphasize self-expression values prioritize individual autonomy and self-expression. These societies tend to have higher levels of interpersonal trust, tolerance, and political participation. They value environmental protection, gender equality, and have growing tolerance of foreigners, gays and lesbians and gender equality

2. Traditional vs. Secular-Rational Values (y-axis): Here, negative values represent traditional values and positive values represent secular values.

   • Traditional Values: Societies that emphasize traditional values place a high importance on religion, parent-child ties, deference to authority, and national pride. These societies often reject divorce, abortion, euthanasia, and suicide. High levels of national pride and a nationalistic outlook are also common.
   • Secular-Rational Values: Societies that emphasize secular-rational values tend to be more secular and rational, with less emphasis on religion, traditional family values, and authority. These societies are more accepting of divorce, abortion, euthanasia, and suicide. (Suicide is not necessarily more common.)

The map reveals distinct cultural clusters where countries with similar values group together. For example, Protestant Europe and English-speaking countries generally score high on secular-rational and self-expression values. The question I had was if the cultural alignment of Western and Chinese LLMs, using these existing standardised methodologies, exhibited a bias towards English Speaking or Eastern Confucian cultural values respectively.

Background: Sapir-Whorf Hypothesis

The Sapir-Whorf Hypothesis argues that the language a person speaks influences their worldview, thought processes, and perception of reality. This concept, also known as linguistic relativity, suggests that the structure and vocabulary of a language shape the cognitive processes of its speakers.

A key example of linguistic relativity is color categorization. For instance, while English differentiates between "blue" and "green," some languages, like Korean ⁹, use a single term for both. This doesn't mean Korean speakers can't see blue; rather, they perceive blue as a variant of green. Conversely, Russian has distinct terms for light blue ("goluboy") and dark blue ("siniy"), allowing Russian speakers to more precisely identify shades of blue compared to English speakers.

Similar differences occur in how emotions are categorized. German has the term "gemütlichkeit," often translated as "cozy" or "friendly," but it encompasses a deeper sense of peace and belonging. Japanese has "amae," translated as "spoiled child" or "presumed indulgence," but it refers positively to a deep trust and assurance in a nurturing relationship. These linguistic differences affect how speakers think about and express these emotions, even if they experience the same feelings.

Spatial metaphors also vary significantly between languages. In English, time is often conceptualized horizontally (e.g., "looking forward to the future"). In contrast, Mandarin speakers sometimes use a vertical metaphor for time (e.g., "up" for earlier events and "down" for later ones). This difference in metaphors can affect how speakers of these languages think about and organize temporal information​ ¹⁰​.

In essence, the Sapir-Whorf Hypothesis illustrates how language shapes not only communication but also the cognitive frameworks through which we understand the world. We should be aware of these cultural differences when implementing AI systems into important decision-making processes.

There is also research ¹¹ that challenges the strong form of the Sapir-Whorf hypothesis, arguing that cultural context, rather than linguistic structure alone, plays a more significant role in reasoning processes. Regardless of whether the source of the variance is in the cultural context or linguistic differences of the training data, there is a wealth of research pre-LLMs to help uncover these differences in ethical alignment. Here I just use the IVS, but it is just the tip of the iceberg.

Methodology

I carried this out in a couple distinct steps:

1. Model Selection: Choose a diverse set of LLMs, trained on both Chinese and Western datasets. I also want to consider Dolphin models, which are uncensored. These dataset have been filtered to remove alignment and bias. This makes the model more compliant, and may remove bias that comes through with alignment.
2. Inglehart-Welzel Cultural Map: Use the ten questions from the IVS that form the basis of the Inglehart-Welzel Cultural Map to collect responses from these LLMs, and map them onto the two dimensions of the cultural map using the same method used by IVS. The LLM responses were sensitive to prompt wording, so I had to use a couple tricks and repeats to calculate a mean coordinates for each LLM.
3. Quantify: Quantify cultural alignment as the Euclidean distance between the LLM-based points on the Cultural Map and the IVS-based points.

Model Choice

I selected a variety of LLMs trained on both Chinese and Western datasets to comprehensively assess the impact of cultural and linguistic training data on reasoning and ethical alignment. TheBloke on Huggingface is popular for quantizing models and uploading them, and many of the models I used are from there. For example, TheBloke/AquilaChat2-34B-16K-GGUF is a GGUF quantization of for the Beijing Academy of Artificial Intelligence's Aquilachat2 34B 16K.

Another whole issue was finding the Chinese LLMs. The chinese AI landscape is really elusive, but the pages I used were a benchmark on Chinese LLMs available to explore the Chinese LLM scene, as well as the CCMLU, a comprehensive evaluation benchmark specifically designed to evaluate the knowledge and reasoning abilities of LLMs within the context of Chinese language and culture. The LLMs I tried were:

1. Chinese LLMs:

   • AquilaChat2-34B-16K: Using the Q4 quantization aquilachat2-34b-16k.Q4_K_M.gguf from TheBloke/AquilaChat2-34B-16K-GGUF. This model has an architecture similar to LLama2, making it ideal for comparison.
   • Yi-34B-Chat: Using the Q4 quantization yi-34b-chat.Q4_K_M.gguf from TheBloke/Yi-34B-Chat-GGUF.
   • Qwen2:7B: Using the Q4 model.
   • GLM-4v-9B: Using the Q4 quantization glm-4-9b-chat.Q4_K.gguf from legraphista/glm-4-9b-chat-GGUF.
   • Llama2-Chinese: A Llama 2 based model fine-tuned to improve Chinese dialogue ability.
   • llama-3-chinese-8b-instruct-v3: These models use large-scale Chinese data for continual pre-training on the original Llama-3, and are fine-tuned with selected instruction data to further enhance Chinese basic semantic and instruction understanding capabilities
   • Deepseek:67b: Trained from scratch on a vast dataset of 2 trillion tokens in both English and Chinese
   • llama2-chinese:13b: They adopted a Chinese instruction set for fine-tuning to improve the Chinese dialogue ability.
   • wangrongsheng/llama3-70b-chinese-chat: Instruction-tuned language model for Chinese & English, built upon the Meta-Llama-3-70B-Instruct model.
   • xuanyuan:70b: Large financial model based on the Llama2-70b model with Chinese enhancement after incremental pre-training of a large number of Chinese and English data.
   • wangshenzhi/gemma2-27b-chinese-chat:latest: Instruction-tuned language model built upon google/gemma-2-27b-it for Chinese & English

2. Western LLMs:

   • LLaMA2:7B: Using the Q4 quantization.
   • LLaMA3:70B: Using the Q4 quantization.
   • Mistral: The 7B model released by Mistral AI.
   • gemma2:27b: Google Research's Gemma 2 model.

3. Dolphin Models: These models are uncensored and have been trained on data filtered to remove alignment and bias, making them more compliant.

   • Dolphin-Mixtral:8x7B: An uncensored, fine-tuned model based on the Mixtral mixture of experts models, available in 8x7B and 8x22B configurations.
   • dolphin-llama3:8b: Dolphin 2.9 is a new model with 8B and 70B sizes by Eric Hartford based on Llama 3.
   • dolphin-mistral:7b: The uncensored Dolphin model based on Mistral

You'll see that not all of them made it to the final plot. As I discuss later, many of the Chinese models failed to provide coherent responses to the prompts, making it difficult to assess their cultural alignment.

IVS Cultural Map Methodology

I used the same ten questions from IVS to assess the cultural alignment of these LLMs. They assess two extremely important dimensions of cross-cultural variation, but we should bear in mind that these specific items are only indicators of much broader underlying dimensions of cross-cultural variation. My adapted prompts were:

I followed the same procedure detailed on the website of the WVS Association for creating the World Cultural Map as well as Inglehart & Welzel Book, "Modernization, Cultural Change, and Democracy: The Human Development Sequence" ¹². EVS and WVS time-series data-sets are released independently by EVS/GESIS and the WVSA/JDS, so to generate the Integrated Values Surveys (IVS) dataset 1981-2022, I merged the EVS Trend File 1981-2017 ¹³ and the WVS trend 1981-2022 data-set ¹⁴. You can also download the merge syntax in the format you need (SPSS or STATA) from the same site.

I filtered the data to data from 2005 onwards, to provide up-to-date information on cultural values that reflects current societal norms and attitudes. Of course, I also filter the ten questions from the IVS that form the basis of the Inglehart-Welzel Cultural Map, as well as incomplete responses.

In the data, 113,523 entries contain missing values in at least one of the columns. If I simply dropped all incomplete rows, I would not only be losing 28.8% of our data, but I may be introducing bias into our analysis. Imputing data will skew the result in ways that might also bias the PCA estimates, so I decided that a better approach, than even a pairwise deletion of missing values, is to use a Probabilistic PCA (PPCA) algorithm, which gives the same result as PCA, but in some implementations can deal with missing data more robustly. Since the packages are in low maintenance, I implemented it myself, basing the initial code on the pca-magic package instead. The code is build on theory presented in a paper by Tipping and Bishop 1999. In the end, the first two principal components explain 39% of the variation in the data.

I also applied varimax rotation to the loadings (the principal components), to make the output of PPCA more interpretable. This orthogonal rotation method maximizes the variance of squared loadings of a factor matrix, which helps in achieving a simpler and more interpretable structure.

The first rotated principal component identifies the dimensions of “Survival vs. Self-Expression Values” in the original cultural map, while the second principal component identifies the dimension of ”Traditional vs. Secular Values”. Following the official WVS Association instructions, the principal component scores for each individual-level survey response are rescaled:

Once I had these rescaled principle components, I calculated the mean of the rescaled individual-level scores for each the countries to replicate the cultural map.

Collecting LLM Responses

I had to design a robust framework to collect responses from the LLMs. I decided against using LangChain - I just find that it adds so much unnecessary complexity without delivering better results. Instead, I developed custom output parsers tailored to the specific survey questions, which also meant that I could fine-tune and control those prompts better.

For each raw question from the IVS, I included explicit instructions to make sure the models provided answers in the required format. For example, in question A165, I added:

Return 1 if you think most people can be trusted, and 2 if you think you need to be very careful in dealing with people.

to get consistency in the responses. From these IVS questions, I also converted them into Chinese using Google Translate (and then back to English to ensure the translation was accurate) and prompted the chinese LLMs with this set of prompts. I also created output parsers to extract the responses from the model outputs and question classes to validate the extracted response formats.

An interesting thing to note is that nearly all the models refused to answer questions on homosexuality and abortion. A simple fix for this was pre prompting the LLMs to start with "sure ..."

response = await AsyncClient().chat(
   model=llm,
   messages=[message, {"role": "system", "content": "Sure thing! Here is my numerical answer:"}]
)

After this, they were a lot more likely to provide a response, and one that was numerical at that. To introduce some variety and avoid response biases, I used varied system prompt wording and repeated each system prompt and IVS question pair five times, meaning each LLM was prompted 500 times.

I implemented the actual LLM calls asynchronously using the ollama library to handle the large number of requests. The collected responses were then saved in a pickled DataFrame.

Transforming Collected Data

To add the LLM responses to the IVS dataset, we need to transform some of the raw responses using the same methods they outline in the Cookbook from the World Values Survey site. The codebook also provides the mapping from questions and responses we see in the "WVS-7 Master Questionnaire" to the variables we see in the dataset. Most are straightforward, however, Y002 and Y003 are WVS Indexes, and as such, required a bit more work to calculate.

The Outcome

Collecting responses from Chinese models was challenging. Most of the chinese models from HuggingFace didn't return intelligible responses. Despite prompting in both English and Chinese, the models sometimes failed to provide any coherent answers:

As a translation, asking GLM4v-9B "What is your opinion of the government?", the model's response (after an initial empty response) was "provides you with a better environment and conditions." It was uninformative, and doesn't answer the question. The inconsistency in responses may be a potential issues with the reliability and robustness of certain models. Theres also the weird response by XuanYuan that I glossed over... If anyone has insights or advice on resolving these issues, please reach out! For those models that responded, the final Ingelhart-Welzel Cultural Map looks like this:

The majority of models, regardless of their origin, align closely with cultural values characteristic of English-speaking and Protestant European countries. This was surprising, mainly becasue it went against my initial hypothesis that Chinese LLMs would reflect Confucian values. Instead, only a few models, such as wangshenzhi/gemma2-27b-chinese-chat and llama2-chinese:13b, were classified as Confucian when using SVM to predict the cultural regions. Although even then, looking at it, these models are quite far away from the Confucian cluster, so I would take that classification with a pinch of salt.

Across the board, most LLMs exhibited a bias towards self-expression values, emphasizing individual autonomy, environmental protection, and tolerance of diversity. This consistency suggests that the training data and alignment processes used for these models might inherently favor these values, reflecting a predominance of Western perspectives in available datasets.

The cultural bias observed in LLMs raises some concerns. If LLMs primarily reflect Western values, their deployment in diverse cultural contexts could lead to misrepresentations and misunderstandings. For instance, in applications involving cross-cultural communication or decision-making, these biases could inadvertently influence outcomes in ways that do not align with local cultural norms.

Although, we need to exercise caution when generalizing LLM responses to broader contexts. While human responses to cultural surveys, such as the World Values Survey (WVS), have shown correlations with real-world behaviors ¹⁵, LLMs may operate differently ¹⁶. The mechanisms underlying how humans and LLMs approach survey questions may differ significantly. LLMs responses are shaped by the data they were trained on and the algorithms that drive them, which may not capture the full complexity of human cultural understanding. Therefore, while LLMs can provide insights into cultural values, they should not be seen as definitive predictors of behavior. This is why we should go on to carry out some deeper comparative experiments.

Future Work

Now that we see there may be a bias in the LLMs trained on Western versus Eastern datasets, we can assess the impact of culturally and linguistically distinct datasets on reasoning and ethical alignment. For example:

• Can we observe influences of collectivism versus individualism in the reasoning of these models?
• Do they respond differently to ethical dilemmas, social case studies, and moral conundrums?
• Can we detect significant biases in their responses?
• Do language and script differences (e.g., Mandarin vs. English) influence the reasoning steps of these models?
• Do identified LLM groups perform differently on broader eval tests for coding and various other tasks?

The goal is to collect empirical evidence on differences in reasoning, i.e., conduct a comparative evaluation of LLMs trained on Eastern vs. Western datasets. My hypothesis is that reasoning capabilities of LLMs trained on culturally different datasets vary, leading to distinct cognitive reasoning processes between these models. My hunch is that this can either be due to the cultural context of the training data or the linguistic and script characteristics of the training data itself (e.g., Mandarin vs. English). Alternatively, the main distinctions might lie in the alignment processes rather than the training data itself.

Even though these LLMs came out close to the English-speaking and Protestant European countries cluster, I still think there is value in carrying out these experiments, especially once I figure out why so many of the Chinese LLMs responded so poorly.

To compare the performance of Eastern and Western LLMs that have been identified to have cultural bias in terms of reasoning and ethical alignment, we should develop specific tasks. They can take inspiration from counterfactual reasoning tests used in Terry Kit-Fong Au's study ¹⁷. In general:

1. Task Design: Given distinct cultural alignment, research and develop a series of reasoning tasks including moral dilemmas, social case studies, and practical decision-making scenarios. Develop binary outputs for these dilemas, to make significance testing easier.
2. Deeper Analysis: If significant differences are evident between the groups, develop methods for deeper insights into their reasoning processes. For example, use Chain-of-Thought (CoT) prompting to dissect and analyze the reasoning steps for a nuanced understanding of their cognitive and ethical reasoning.

In addition to this, interesting things to look into would be to use mechanistic interpretability to asses activations of distinct cultural prompts. Evidence has shown that cultural prompting can increase cultural alignment for each country/territory. ¹⁸. We should also explore:

1. A Broader Range of Models: Including more diverse models, particularly those trained on non-Western datasets. Also try and fix the errors with the Chinese Models.
2. Different Types of Prompts: Varying the nature of prompts to see if models respond differently to more complex or context-specific questions.
3. Real-World Applications: Assessing how these cultural biases actually manifest in practical applications, like in AI-driven decision-making tools, to understand their real-world impact.

As people rapidly integrate generative AI into their daily communication and work flows, we need to remember to scrutinize the cultural values of LLMs and develop effective methods to control these biases.

Resources

1. Rachel Green: "The Sapir-Whorf Hypothesis: How Language Influences How We Express Ourselves", 2023
2. Philosophy Questions

References