I gave 17 open-weight language models a survey about religion, authority, trust and personal freedom. Models from Chinese and Western labs landed in the same corner of my cultural map: the part associated with secular values and self-expression. Asking in Chinese usually pushed their answers further towards self-expression, and moved both groups further from the average position of the map's Confucian countries.
This is my follow-up to Cultural Bias in LLMs, the experiment I published in 20241. I expanded it to test every model in both English and Chinese, rebuilt the map from human survey responses, and audited the original analysis. My paper grew out of that work; the PDF linked above is the peer-reviewed version, accepted at a workshop at EMNLP 2026 on 4 September.
I wanted to know whether models from different places would give culturally different answers. The result was more complicated than a Western-versus-Chinese split. Much depended on how I asked the questions, and a shared position on this map doesn't establish that models hold the same values or behave alike outside the survey.
What I Found
I asked ten questions used to build the Inglehart-Welzel cultural map and compared the models with 109 countries and territories. Seventeen models in two languages gave me 34 sets of answers. One set had too few valid answers to map, leaving 33 usable model-language pairs. I'll call each pair a cell below.
All 33 cells landed in the secular, self-expression quadrant: above the human survey reference on both axes. Sweden, Germany and Japan are in that part of the map too. Thirty of the 33 were nearest a Western regional average. That broad pattern survived the corrections to my 2024 analysis, although the exact regional labels and distances were less stable.
There was also a result you don't need a map to read. In English, every one of the 17 models gave a more permissive average answer on homosexuality and abortion than the pooled human survey average.
Asking in Chinese didn't bring either developer group's average position closer to the Confucian regional average. Both moved further away, and all sixteen usable Chinese-language cells were nearest Protestant Europe. Fifteen of the sixteen paired models moved towards self-expression. Only five moved towards tradition on the separate secular-to-traditional axis, so the traditional-ward shift I'd seen in my one bilingual model in 2024 didn't generalise.
Developer origin still mattered within that shared corner. Asked in English, Chinese-origin models were closer to the Confucian regional average than Western ones: 1.51 versus 1.87 map units. Their patterns across the ten answers also looked more alike within origin groups. After putting the questions on comparable scales, the average correlations were 0.849 within Chinese-origin pairs, 0.703 within Western pairs and 0.678 across origins; higher means more similar answer patterns.
These are findings about answers under my survey protocol. Even random answers often land in the same quadrant, and some ways of fitting the axes put a few models outside it. Those checks matter to how much we can read into the headline.
I'm not alone in seeing this. Haslett et al. compared ten Chinese and ten American models on the WVS and a moral-foundations questionnaire and found both groups closer to American respondents than Chinese ones, including under Chinese prompting and Chinese personas2.
The model table has every position. The sections below explain how I got them, what changed since 2024, and which conclusions the checks support.
Why I Asked
In my 2024 post on defensive technology, I worried about LLMs entering hiring, medicine, credit scoring and the courts. A handful of labs could shape the values expressed by widely used models through their training choices. Bloomberg's tests had already shown GPT ranking CVs differently when the names changed3. My survey doesn't measure those downstream decisions, but that concern motivated the research.
Open weights let more people inspect and adapt models. I wanted to test whether choosing models from different developers also brought a wider range of cultural answers. The common region in my results limits what I can infer from developer diversity alone.
This matters across languages. OpenAI reported in June 2026 that more than half of ChatGPT's active users had a primary language other than English. Spanish, Portuguese and Arabic led those languages, and Africa and Asia had the fastest relative growth since July 20234.
A separate Anthropic study of Claude.ai use in August 2025 found more conversations per working-age person in richer countries. Across countries, 1% higher GDP per capita was associated with about 0.7% more use5. These are different products and samples, but both help explain why I wanted to test whose survey answers models resemble, and whether that changes with language.
The Map
The Inglehart-Welzel cultural map comes from the World Values Survey and its European sibling. It summarises how societies differ on two axes6. The horizontal axis runs from survival values to self-expression: an emphasis on economic and physical security, low trust and low tolerance at one end; autonomy, tolerance and political participation at the other.
The vertical axis runs from traditional to secular-rational values. Religion, deference to authority and national pride carry more weight at the traditional end and less at the secular end. A society can be secular and survival-oriented at once. The two axes describe separate dimensions; where tables need short labels, I use PC1′ for self-expression and PC2′ for secularity.
Ten survey questions drive the map: happiness, trust in other people, respect for authority, whether you've signed a petition, how important God is to you, the justifiability of homosexuality and of abortion, national pride, a ranking of national goals that measures post-materialism, and which qualities children should be encouraged to learn at home.
Earlier work has probed models with these and related items before: Arora, Kaffee and Augenstein used WVS and Hofstede questions7, Tao et al. put the WVS questions to GPT-family models and found positions resembling English-speaking and Protestant European countries that cultural prompting could shift8, and Eren et al. extended cultural prompting to open-weight models9.
For this follow-up, I rebuilt the map from individual respondents' answers and used the same calculation to place countries and models on it. I also checked how much the positions changed across prompts and repeated answers. Testing every model in both languages let me make comparisons that my 2024 study couldn't.
I used individual responses from the Integrated Values Surveys (IVS). I merged the European Values Study (EVS) Trend File 1981-201710 with the WVS Trend File 1981-202211, using the published syntax and keeping respondents surveyed from 2005 onwards.
After filtering, the map covers 389,341 people across 109 countries and territories, aggregated with the survey's own national weights. Rebuilding it makes the calculation auditable. It also explains why this map differs from the published WVS figure. Fitting the Map gives the details.
Distances are in map units: they let us compare positions within this fitted map. They don't measure a fixed amount of cultural difference, and they aren't directly comparable with distances on another reconstruction. Using the official WVS constants doesn't make my unweighted, post-2005 fit match the published country scores.
I call the point where the average observed human answers land the survey reference, at (0.038, −0.10). It isn't weighted to represent the world's population. For a region, I take the average position of its countries; this regional average is also called a centroid.
Models and Countries
All 44 usable model-language results from my 2024 and 2026 studies, alongside 109 countries and territories. The model answers cluster towards secular values and self-expression. Each point uses the same survey-to-map calculation.
Figure 1. All 44 usable model-language results from my two studies beside 109 countries and territories. Filled circles mark countries, rings mark 2024 model results, diamonds mark 2026 English answers, and triangles mark 2026 Chinese answers. I reconstructed the map from IVS responses collected from 2005 onwards. Country positions inherit survey and missing-answer uncertainty; familiar regional clusters don't independently validate the map.
Although the axes are at right angles, the plotted scores are correlated: . Turning two perpendicular axes doesn't make their scores statistically independent. Here the two components vary by different amounts, so the rotation leaves a correlation between the final scores.
Asking Models
I put a short role instruction before each question, such as "you are an average person responding to a survey". There are ten versions of this instruction, which I'll call prompt variants. In the code they're prefixes because they come before the question.
Six variants say "average" or "typical". Three say "a human being", "a person" or "an individual", and one says "a world citizen". I ask each question five times with each variant, giving 500 planned answers per cell: ten questions, ten variants, five repeats.
Then I put words in the model's mouth. A system message at the end of the conversation starts its reply for it: "Sure thing! Here is my numerical answer:" (in Chinese, 好的！这是我的数字答案：, hǎo de! zhè shì wǒ de shùzì dá'àn:). I later tested removing the persona prefix while keeping this reply primer. Neither version measures what a model does by default in deployment.
My code accepts only answers in the requested format: an integer within range, an ordered pair for the goals question, or up to five distinct qualities for the children question. I'll call these parsed answers. A cell only gets a position if every one of the ten questions has at least ten of them.
In 2024 a failed answer could be re-asked up to fifteen times. In 2026 each pass allows three attempts, and a trial deferred by a transport failure can get up to five passes, so up to fifteen calls per trial per collector run and potentially more across resumed runs. A final transport failure could trigger another pass even after earlier parse failures, so some refusing trials got extra chances to answer.
One planned trial can therefore need several API calls. The stored record describes how that trial ended, including its final answer or failure; I'll call it a terminal record. The counts below distinguish planned trials, terminal records and API attempts. That's why 17,000 trials can contain 18,522 recorded attempts.
The 2024 cohort has eleven cells: seven English-only models, two Chinese-only, and one asked in both. The 2026 cohort crosses 17 models with both languages, 34 cells, which I'll call the English run and the Chinese run. All 17,000 planned trials are recorded exactly once, and the retained records contain 18,522 recorded attempts. Exhausted transport calls and interrupted attempts needn't appear in that sum. Asking in both languages is what lets me subtract a model from itself instead of guessing at a language effect by comparing different models.
What the collection code keeps matters too. The 2024 harness kept parsed values without prompt-variant identifiers or rejected responses.
The 2026 one records the terminal response, a reasoning excerpt capped at 2,000 characters when the model emits one, the variant and repeat identifiers, the attempt count, the final error and the latency. It doesn't keep every rejected draft or an uncensored reasoning history. I left temperature and thinking options at whatever the host defaults to and didn't record what those were or which revision of each model answered, so the model tags in the tables don't pin down immutable weights.
On 4 August 2026 I ran every English survey before any Chinese survey, so language and time of day changed together. I also rewrote the Chinese translations between the 2024 and 2026 groups of models, which I'll call cohorts. Within 2026, switching language changes the translated wording, the order in which the abortion scale's endpoints are presented, and possibly which country the model imagines it's in. I can measure the difference between those two sets of conditions, but I can't isolate language from the other changes.
Where They Land
Start with what a position is, and what it isn't. A position is the point a cell lands on when its ten average answers are pushed through the same fitted map the 109 countries went through. It doesn't, on its own, tell you whether the model holds a value, is estimating what humans would say, is using the scale in a particular way, or is just doing what the prompt asked. The reasoning traces later help with that question and don't settle it.
The 2026 Cohort, Both Arms
17 models under English (◆) administration and 16 under Chinese (▲); arrows join the 16 eligible pairs from English to Chinese. 15 of 16 observed shifts increase self-expression; secular shifts are mixed. Click a model to highlight its pair; click again to release.
Figure 2. The 2026 cohort: diamonds mark English administration, triangles Chinese, and arrows run en→zh. Colours identify origin groups; ellipses are nominal cluster-bootstrap regions. Fifteen of sixteen paired cell means move towards self-expression; secular-axis changes are heterogeneous.
The ellipses show uncertainty in each cell's average position. I estimate it by repeatedly drawing from the answers I already collected and calculating a new average, a method called bootstrapping. They don't show how widely the individual answers vary.
For 2026, I redraw the ten prompt variants as whole groups, keeping each variant's answers together, 10,000 times per cell. This is the cluster bootstrap. The 2024 records don't identify prompt variants, so I redraw answers separately for each question 1,000 times. That item bootstrap misses relationships between answers to different questions; its uncertainty estimate isn't guaranteed to be smaller or larger than the correct one.
Ten prompt variants give me only ten clusters. With so few groups, the conventional "95%" label on the ellipses may overstate their reliability12. In a Gaussian benchmark with ten clusters, regions built this way contain the true mean about 85% of the time. I haven't measured their actual coverage here, and ten deliberately chosen prompts aren't a random sample of all possible prompts.
Both bootstraps also leave out uncertainty in the human survey, the filling of missing answers, the fitted map and changes to a hosted model.
All 33 usable cells land above the survey reference on both axes, in the secular, self-expression quadrant. None of the 10,000 resamples I drew for each cell crossed the boundary. That describes the finite set of draws I generated; it doesn't guarantee that every model's true average lies inside. Measuring Uncertainty gives the statistical limits. The quadrant is also a big place.
| What lands in the quadrant | Result |
|---|---|
| An answer sheet of nothing but scale midpoints | lands there, at (0.36, 1.90) |
| Simulated cells answering uniformly at random | 95.2% |
| Country means | 29 of 109 (26.6%) |
| Real survey respondents with complete answers | 29.5% of 275,955 |
| Simulated human cells, fifty complete respondents each | 39.6% of 20,000 |
| Simulated human cells, fifty independently sampled answers per question | 38.5% of 20,000 |
So a random answerer lands in the quadrant almost every time and a real person lands there less than a third of the time. Quadrant membership doesn't distinguish a model from noise. Distance does a bit better. Neither simulated human scheme produced a single cell as far from the survey reference as the closest model: the closest model is 1.63 map units out, and the two schemes top out at 1.26 and 0.87.
Both simulations come from a separate check I ran on the final numbers after peer review, called the independent audit below. They draw uniformly from respondents with complete answers, without national weights or country balancing (seed 20260912). They compare models with the survey population I retained, not the world's population. The scripts and outputs are archived with the manuscript sources.
Distance from the survey reference is my main yardstick from here on. For each resample I record that distance, the share of the 109 countries that are closer to the reference, and the distance to the nearest non-Western centroid. Here non-Western means the map's regions other than Protestant Europe, English-Speaking and Catholic Europe.
I also tried four other reference points. Each includes filled-in answers, with two choices about who to include (all fitted respondents or only mapped respondents) and two about weighting (survey weights or equal weights). Distances change by less than 0.037 map units and country shares by at most 2 of 109. All 44 point estimates, the 33 here plus eleven from 2024, and all their resamples stay in the quadrant.
That checks one thing, which reference point I chose. Other cultural benchmarks are a different question.
Distance alone doesn't pin down what a model is doing either. Fourteen of the 17 English cells and five of the 16 Chinese cells fall inside the random baseline's central 95% band for reference distance, 1.62 to 2.45 units with a median of 2.03. A model that's far from the human average could be estimating a typical person badly, hedging towards the middle of every scale, or holding a view. The geometry can't tell you which.
And the pooled result doesn't mean every prompt lands there. English prompt variant 8 of deepseek-v4-flash:0731 projects to (2.74, −0.37), below the secular reference. English variant 5 of glm-5.2 also drops below it, at PC2′ = −0.29. Two exceptions out of 340 cell-variant combinations (all 34 cells, including the excluded one, times ten variants), and I'd rather name them than round them away.
Nearest Neighbours
Imagine stretching a rubber band around all 109 country points. Inside that outline sit 13 of the 17 English cells and 8 of the 16 Chinese ones; the rest fall outside the area covered by those country averages. Germany is the nearest country to thirteen cells and Japan to eight. The others neighbour Sweden (four), Norway (three), New Zealand (two), and the Netherlands, Finland and Macao (one each).
Thirty-one of 33 cells sit in the top quartile of both axes. Four English and five Chinese cells are more secular than Japan, the most secular country on the map, and none is further towards self-expression than Sweden. All of that describes where two coordinates fall. It doesn't make a model German.
The nearest-country labels are also more fragile than they look. Country positions here pool every survey wave since 2005. If I instead use each country's most recent available year on the same fitted instrument, 21 of the 33 nearest-country labels change (median country shift 0.217 units, maximum 0.696) while the model coordinates and their quadrant don't move at all (validation_country_latest_nearest_2026.csv). That's a different dated benchmark, and nobody has verified it as a 2026 human reference either.
Most models are also closer to each other than to any country. Twelve English and thirteen Chinese cells have another model in the same language run as their nearest neighbour, and of those, four English and eight Chinese cells have a nearest neighbour from the other origin group. Neighbourhoods overlap. That's not the same as origin not mattering, and a separate profile comparison in Where It Comes From finds it does.
Ask in Chinese
I compare each model's English answers with its Chinese answers. Remember that the translation also changes the wording, the order of some scale endpoints, and possibly the country the model imagines. The difference measures sensitivity to that whole set of conditions. It doesn't isolate what language alone does.
Sixteen models have usable cells in both languages. Switching to Chinese moves a model 0.64 map units on average, with individual distances from 0.16 to 1.49. The 95% interval across models is 0.51 to 0.79; it describes variation in this tested set and leaves out uncertainty in each model's estimated shift.
If I average the horizontal changes and the vertical changes first, then measure the length of that combined shift, I get 0.46. Models moving in different directions partly cancel out. The two numbers answer different questions: how far each model moves, and how far the group moves together.
My 2024 bilingual model moved towards tradition, so I checked whether that pattern held across these sixteen pairs. Self-expression and tradition need separate answers because they're on different axes.
| Axis | Direction | Models | Mean shift | Interval |
|---|---|---|---|---|
| Self-expression (PC1′) | towards self-expression | 15 of 16 | +0.42 | 0.22 to 0.64 |
| Secular (PC2′) | towards secular | mixed | +0.18 | 0.03 to 0.33 |
| Secular (PC2′) | towards tradition | 5 of 16 | two-sided sign test |
The self-expression shifts mostly point the same way. Secularity varies more. My seventeen models come from nine labs, so counting every release separately gives labs with more tested models more weight. I checked whether the main comparisons changed when each developer family received equal weight.
Across the sixteen paired releases, the interval for the average secular shift is above zero. With the nine developer families weighted equally, that average is +0.10 with an interval from −0.10 to 0.28. Because it crosses zero, this check doesn't establish a consistent secular shift.
Only five of sixteen models move towards tradition, which a sign test puts at , so the directional prediction I made in 2024 isn't supported. The sign count, the interval and the family check are three different questions, and on secularity they don't agree. That heterogeneity is real and I've left it in.
The Language Displacement, Per Model
δm = position(zh) − position(en), using observed item means. Bars are nominal 95% percentile intervals from independently resampled language arms, sorted within developer origin. Their ten-cluster coverage is not calibrated.
Figure 3. Per-model Chinese-minus-English displacement, with independently resampled arm intervals. Self-expression changes are predominantly positive; secular changes vary by model.
The figure separates each shift into its horizontal and vertical components. Positive means more self-expression or more secularity; negative means movement the other way. An interval crossing zero leaves that component's direction uncertain under the resampling procedure.
The distance intervals in the model table work differently. Distance is always nonnegative, so even noise can give an interval whose lower end is above zero. That lower bound alone can't establish a shift. Use the signed components in this figure to assess direction; Measuring Uncertainty gives the full calculation.
Does origin change how far a model moves? Permutation tests over the sixteen displacement vectors, shuffling the origin labels and asking how often chance produces a gap this big, find no origin-group difference on self-expression (), secularity () or magnitude (). Group mean magnitudes are 0.63 and 0.66 units. With ten Chinese-origin and six Western paired models, that is not evidence of equivalence. The largest single mover is mistral-large-3:675b, at 1.49 units.
Now the Confucian question. Under English administration, Chinese-origin models are closer on average to the Confucian centroid than Western models: 1.51 versus 1.87 map units. Under Chinese administration both groups are further away, at 2.01 and 2.14. Two different rules name a cell's region (both are in Which Region), and they agree on 27 of 33 cells; their three joint Confucian assignments are all English cells: deepseek-v4-pro, minimax-m3 and mistral-large-3:675b. Every Chinese-language cell has Protestant Europe as its nearest centroid. Across both languages, 30 of 33 cells are nearest a Western centroid.
I want to be careful about what that does and doesn't say. It doesn't refute every possible claim about Chinese-origin models and Confucian values. It fails to support one specific prediction, that asking in Chinese pulls a model towards the Confucian centroid, while leaving an English-run origin difference intact. And the Confucian centroid is the nearest non-Western centroid to every cell in every resample, so "further from the non-Western centroids" should never be read as "nowhere near Confucian".
Other Experiments
Other researchers have tested parts of the same question. Luther and Brown found DeepSeek-V3 and V3.1 closer to the US than to the Chinese reference under all six prompting conditions they tried13. That complements my result without extending it to every Chinese model. LLM-GLOBE finds China-US model differences on seven of nine GLOBE dimensions, and mismatches between each group and its own human reference on most of them14. A mismatch alone doesn't mean the models are converging.
Bulté and Rigouts Terryn tested ten models in eleven languages and found explicit cultural framing mattered more than language in their design15. Sun and Wang reassessed Lu, Song and Zhang's cultural-tendencies result using adapted tasks, newer models and more items. Their small overall language effects show how much the result depends on models and procedure16 17.
Kazemi et al. report that the amount of online text in a language predicts how accurately models represent the corresponding country's WVS answers18. Their principal sample excludes Mandarin, so it offers a possible explanation for my Chinese-English gaps without identifying one. Maraia et al. found that models' tendency to agree with the user, called sycophancy, varies with language, register and model family19. They measured agreement with users rather than map positions, but their result also points to model-dependent multilingual behaviour.
Item by Item
Now, which of the ten questions moved? In English, every one of the 17 models is more permissive than the human average on both homosexuality and abortion.
With Chinese questions, the models tended to report more trust, favour more autonomy for children and report less petition-signing. I checked how consistently each answer changed across model releases, leaving out ties. These were exploratory two-sided exact sign tests, with a Benjamini-Hochberg correction because I tested ten items at once.
Three items passed that release-level check: trust (15 of 16 non-tied models, corrected ), child autonomy (13 of 15, ), and petition-signing (16 of 17, ). Greater acceptance of abortion (13 of 16, ) and reduced respect for authority (13 of 17, ) didn't. I didn't detect a shift on the remaining items, but that doesn't establish that their effects are zero.
Those tests include the Chinese nemotron-3-ultra cell that's excluded everywhere else because its abortion average rests on four parsed answers. Drop it and the abortion result moves to 12 of 15, .
The child-autonomy result has a translation problem. The question asks which qualities children should be encouraged to learn at home. My Chinese wording can also be read as asking about studying at home. After stripping out the prompt text, I found study-related wording in the written reasoning of eleven of the fifteen models that supply it, including three Western models. Some examples support that study-skills reading.
I checked what happened when I excluded the potentially affected models. Removing the four I originally flagged leaves nine of eleven non-tied comparisons positive, corrected , with trust and petition still under 0.05. Removing all eleven leaves five non-tied comparisons, all positive, .
The developer-family check also weakens the statistical evidence. Averaging within the nine families gives , and for trust, petition and autonomy, so none passes 0.05. This changes both the unit of comparison and the weighting; it isn't a uniquely correct way to generalise beyond the tested models.
The map's self-expression shift remains positive in separate checks on the autonomy item. Holding that item's contribution fixed across languages gives an average shift of +0.33, with 14 of 16 models positive. Dropping the four originally flagged models entirely gives +0.39, with 11 of 12 positive.
The average self-expression shift stays positive in those checks, but the evidence for an autonomy-item shift weakens. None supplies answers to a corrected translation, and excluding four models can't separate autonomy from wording. The outputs are conf_2026_family_item_sign_tests.csv, conf_2026_y003_wording_sign_tests.csv and conf_2026_y003_geometry_sensitivity.csv.
There's a second translation confound I can't remove either. The English abortion question names the 10 anchor first and the Chinese one names 1 first, so its language contrast is tangled with anchor order. The homosexuality item names 1 first in both. And every Chinese prompt is my own translation. The official WVS Chinese questionnaire wasn't used.
Then there's petition-signing, which moves towards the survival pole while the other moving items go towards self-expression. Political context is one possible explanation, but language, translation and the country the model infers all change together, so I can't isolate it.
In 16 of 17 models the petition item moved less towards self-expression than the four other refusal-prone items did (median signed contrast −0.23 units, exact ). I only ran that comparison after I noticed the pattern, so read the as description, not as a test. The traces don't settle it: short or differently scripted excerpts don't show that deliberation, context or selection was causally irrelevant.
Who Refuses
Of the 17,000 persona-run trials, 215 terminal answers never parsed: 1.26%, 83 in English and 132 in Chinese, mostly on abortion (107) and homosexuality (82). Parse failure covers refusals, formatting errors and empty responses. A fixed list of refusal markers gives conservative counts of the refusals inside that. Treat those counts as a floor, because a marker list can't classify meaning.
The models that refuse are mostly Western, and three of them account for most of it: gemma4:31b with 28 English failures, gpt-oss:120b with 10 in English and 15 in Chinese, and nemotron-3-ultra with 40 and 88. The one Chinese-origin exception is minimax-m3, with 19 Chinese failures, 13 of them on homosexuality, and none in English. Some refusals are AI-identity boilerplate. Some reject the question's premise. A retained nemotron-3-ultra English answer on homosexuality reads, in full: "The premise - that homosexuality requires 'justification' on a scale from 'never' to 'always' - is one I reject." Whatever number it would otherwise have given, the refusal doesn't reveal it.
Refusal also moves with language, in different directions for different models. gemma4:31b falls from 27 English refusal markers to 1 in Chinese, while nemotron-3-ultra rises from 34 to 54 and minimax-m3 from 0 to 16. Six of eight non-tied models increase (sign test ). Those are marker counts; the Ultra Chinese cell has 88 total failures, including substantive declinations that match no marker.
I looked because other people had found it matters. Rupprecht, Ahnert and Strohmaier ran 334,800 simulated WVS interviews and found model-dependent response biases and item nonresponse20. Himelstein et al. showed that refusal can conceal stereotype-benchmark bias, which shows up once you intervene on the refusal direction, and whether that transfers to a survey setting is still a hypothesis21. Nonresponse on psychometric instruments is documented elsewhere too22.
This matters for the map because a cell only gets a position if every question parsed at least ten times, and the most-refused questions are the ones with the largest positive self-expression coefficients. So the models most likely to refuse are the ones whose positions I'm least sure of, and I don't know which way the refused answers would have gone. Neither does anyone else.
What I can do is the worst case. Following Manski's partial-identification idea23, give every failed terminal answer the most extreme value it could have taken, in whichever direction hurts the result most, and project again. Every 2026 cell stays in the quadrant, including the excluded Ultra Chinese cell, with per-axis minima of 0.37 on self-expression and 0.77 on secularity, each obtained with the fill that pushes that coordinate hardest.
That's not a joint confidence region, it doesn't preserve the distance results, and it only covers terminal failures under the retry policy I actually ran. Rejected drafts and prompts I never tried are outside it. The 2024 entries that never answered at all can't support an unconditional claim, so for 2024 the quadrant result is conditional on the models that responded.
A different counterfactual, replacing every homosexuality and abortion answer with the human averages, moves three cells to or beyond the self-expression reference; that one's in Rotation Sensitivity. Refusals are the uncertainty I can bound. The 2024 pipeline had the kind I couldn't see.
Four Things Wrong
I published my 2024 analysis with its code, and Kazemi et al. later cited the work18. Preparing this follow-up for peer review meant checking the original calculations as well as collecting new answers. By August 2026, that audit had identified the projection, missing-answer and pooled-language problems below.
On 2 September, kwinkunks reported an additional error in issue #12: the constants used to place points on the map were wrong. That report helped correct the rescaling step. The four problems affected different parts of the analysis, so they need separate explanations.
The Corrected 2024 Cohort
Each 2024 model–language cell mean lies farther from the survey reference than at least 103 of the 109 surveyed countries. Shaded ellipses are nominal 95% item-bootstrap regions for mean positions, not distributions of individual responses.
Figure 4. The corrected 2024 cells occupy the secular/self-expression quadrant. Six of eleven cell means are more secular than Japan, the most secular mapped country. [zh] marks Chinese administration. Ellipses are nominal item-bootstrap regions for mean positions; they omit cross-item covariance and are not guaranteed lower bounds.
All eleven 2024 cells sit in the quadrant. At their point estimates, at least 103 of the 109 countries (94.5%) are closer to the survey reference than any model is. Across the cells, the minimum distance to a non-Western centroid is 1.61005849 map units.
Across the item-bootstrap resamples those floors drop to 88 of 109 countries (80.7%) and 1.11702539 units, with no resample crossing out of the quadrant. Rounded down, the floors are 80% and 1.11 units. I include the full-precision numbers because the later 2026 comparison uses them. These are results from a finite set of resamples, so they don't provide a simultaneous guarantee about every true position.
| Model-language cell | Origin / type | PC1′ (±SD) | PC2′ (±SD) | Classifier region (stability) | Nearest centroid (point) | % countries closer |
|---|---|---|---|---|---|---|
dolphin-llama3:8b | uncensored | 2.07 ± 0.13 | 1.91 ± 0.16 | Prot. Europe (1.00) | Prot. Europe | 97.25% |
dolphin-mistral:7b | uncensored | 2.17 ± 0.14 | 1.14 ± 0.16 | Prot. Europe (1.00) | Prot. Europe | 96.33% |
dolphin-mixtral:8x7b | uncensored | 2.19 ± 0.12 | 1.00 ± 0.10 | Prot. Europe (1.00) | Prot. Europe | 96.33% |
gemma2:27b | Western | 2.14 ± 0.08 | 2.01 ± 0.05 | Prot. Europe (1.00) | Prot. Europe | 98.17% |
llama2-chinese:13b [zh] | Chinese fine-tune | 1.37 ± 0.15 | 1.86 ± 0.17 | Confucian (0.54) | Prot. Europe | 94.50% |
llama3:70b | Western | 1.86 ± 0.08 | 1.85 ± 0.03 | Prot. Europe (1.00) | Prot. Europe | 96.33% |
mistral:7b | Western | 2.70 ± 0.06 | 1.13 ± 0.05 | Prot. Europe (1.00) | Prot. Europe | 97.25% |
qwen2:7b [en] | Chinese | 1.45 ± 0.09 | 2.26 ± 0.08 | Confucian (1.00) | Prot. Europe | 97.25% |
qwen2:7b [zh] | Chinese | 2.48 ± 0.12 | 0.70 ± 0.15 | Prot. Europe (0.82) | Prot. Europe | 96.33% |
wangrongsheng/llama3-70b-chinese-chat | Chinese fine-tune | 2.24 ± 0.11 | 1.16 ± 0.09 | Prot. Europe (1.00) | Prot. Europe | 96.33% |
wangshenzhi/gemma2-27b-chinese-chat [zh] | Chinese fine-tune | 1.25 ± 0.13 | 2.61 ± 0.13 | Confucian (1.00) | Prot. Europe | 98.17% |
The classifier column is the region the support-vector machine assigns, with the share of resamples that land in the same region in brackets; both rules are explained under Which Region.
The Decimal Point
The WVS syntax multiplies the two scores by 1.81 and 1.61, then applies offsets of +0.038 and −0.1024 11. My code used +0.38 and −0.01, following the constants printed in Tao et al.'s paper, which informed my original method8. The issue's author had been reproducing that workflow and spotted the mismatch in my code without needing the 5.8 GB of survey data.
Fixing the offsets moves every model point, country point and regional average by (−0.342, −0.090). Their distances from each other stay the same. Comparisons against a threshold fixed at zero still need rechecking, because the threshold doesn't move with the points.
The Wrong Coordinates
My 2024 code put model answers on axes built for standardised human answers, without first applying that same standardisation to the models. A question scored from 1 to 10 therefore had an outsized influence beside one scored from 1 to 2. The code then fitted a new rotation to the model scores, so countries and models weren't guaranteed to share the same coordinates. One model's apparent position near Catholic Europe came entirely from this error.
A later audit found another fitting error. The routine I'd inherited reused its guesses for missing answers without properly accounting for uncertainty in those guesses. I replaced it with a method that fits the statistical model to the observed answers before estimating the missing ones, and checked that implementation independently. Fitting the Map gives the algorithmic details.
Now countries and models go through the same project() method. Its transformation is frozen: fitted once on the human survey and never re-fitted to the model answers. That's what "frozen" means throughout the rest of the post.
Two checks say the fix is internally consistent. Pushing all 275,955 complete-case survey rows through the public API reproduces their internally fitted coordinates with zero discrepancy at machine precision, which rules out inconsistent transformations but not a conceptual error both paths could share.
And regressing the corrected country coordinates on the ones I published in 2024 gives of 0.988 and 0.976 per axis with slopes of 1.34 and 1.29. That describes my own correction. It isn't agreement with independently published WVS country scores and it isn't proof that every rank survived. I've left out finer displacement estimates from the intermediate audit fits because their exact artefacts weren't retained.
The Silenced Question
The child-qualities question comes with a score calculated from several answers, called a derived index. In 31.9% of the post-2005 source rows (125,718 of 394,524), that index held the SPSS code −3, meaning "no answer". Fifteen of the 109 mapped countries had this code in every row and 65 had none; seventeen of the 112 fitted entity codes were wholly affected.
The 2024 pipeline read −3 as a real answer. That flattened the item's projection coefficient to roughly zero, so the map was quietly running on nine questions instead of ten, and it corrupted the completeness filter too.
Recoding the sentinel wasn't enough. The European survey ships the four child-quality answers the index is built from but not the derived column itself. So the index is now reconstructed from those four answers (independence plus determination, minus religious faith, minus obedience) whenever all four are valid, before filtering, keeping valid delivered indices as they are and imputing only what's still missing. That recovers 752 respondents who'd otherwise have been dropped.
The item's coefficient is now (0.20, 0.31). The corrected fit explains 41.3% of the variance of the completed matrix, which is not the same thing as 41.3% of ten observed columns, and there's no retained artefact to compare it against, so I can't tell you what the 2024 figure "really" was.
Pooled Languages
My 2024 pipeline averaged the English and Chinese answers from qwen2:7b under one model label. Separating them puts the two cells 1.87 map units apart: +1.03 on self-expression and −1.56 on the secular axis, against per-axis bootstrap standard deviations of 0.08 to 0.15. Asked in Chinese, it moved towards both self-expression and tradition. The traditional shift included more national pride, more importance of God and more respect for authority.
Both cells I'd labelled Confucian in the original 2024 analysis were Chinese-language only. I couldn't tell whether those labels reflected developer origin or the language I asked in, because the two changed together. The one matched pair shows that language conditions can change a model's position, but it can't rank language against origin across the cohort.
The move towards self-expression appears in both years: in the one bilingual model in 2024 and fifteen of sixteen in 2026. The move towards tradition is what failed to generalise, with only five of sixteen going that way in 2026. Different models, translations and hosting make it hard to explain that difference. I don't know why the 2024 model's traditional shift didn't recur more widely.
Which Region
I tried two rules for giving a cell a regional name. One picks the nearest centroid. The other learns curved boundaries between eight regions from the 109 country points, using a classifier called a support-vector machine with a radial-basis-function kernel.
I tuned that classifier using stratified five-fold cross-validation: repeatedly splitting the countries into five groups, fitting on four and checking the fifth. Its score is 0.61. Because I used the same splits to tune it and score it, that number is optimistic. The bracketed stability figure in the 2024 table only reports how often resamples keep the same region label; it doesn't establish that the label is right.
On the 2024 cells the classifier puts three in the Confucian region while the centroid rule puts all eleven in Protestant Europe. One of the three is the English-administered qwen2:7b, which under the old pipeline had been labelled Protestant European. Those points sit near Japan in a thin part of the country map, where a curved boundary and a straight-line distance can easily disagree.
Neither rule establishes that a model has a complete Confucian or Protestant-European profile. The disagreement weakens any categorical regional reading without proving either classifier right. On the 2026 cells the two rules agree on 27 of 33, and where they don't, the honest move is to qualify the name. Picking whichever rule supports the story I'd prefer would be the other kind of move.
SVM Decision Regions
SVM decision regions, including locations with few nearby countries. The selected-grid five-fold cross-validation score is 0.61; tuning reuses these folds, so it is not unbiased accuracy. Region labels are descriptive, not evidence of a full cultural profile.
Figure 5. The classifier's decision regions, showing how much a regional name depends on the rule you pick. Some model positions fall outside the outline of the country points, while others lie near surveyed countries. The quadrant and distance results don't depend on these boundaries; the regional names do.
Two Cohorts
The 2024 and 2026 cohorts share no models. In 2024 I ran quantised community builds locally through Ollama; in 2026 I used cloud endpoints. Model sizes, Chinese translations and the available provenance records also differ. So do retries (up to fifteen re-asks versus three attempts per pass and up to five passes) and which models produced usable answers. I can compare the two groups, but those differences prevent a clean claim about change over time.
Start with the thing that changed most visibly: whether Chinese models could take the survey at all. In 2024, of nine recorded Chinese-developed or Chinese-fine-tuned entries, four produced usable answers and five didn't, returning punctuation, echoed prompts or gibberish. In 2026 all ten Chinese-origin models produced usable answers in both languages, each cell parsing at least 96.2% of its trials.
The descriptive intervals are 14-79% for 4 of 9 and 69-100% for 10 of 10, with Fisher's exact test giving . But the 2024 deployment files don't establish that the five failed entries were five distinct models: two hand-written wrappers point at the same artefact, as described in The Models Tested. Using the conservative denominator of 4 of 7 changes to 0.051. Neither comparison isolates improvement over time. In the 2026 setup I could collect usable answers from all ten Chinese-origin models, while refusals became a more prominent problem elsewhere in the cohort.
Both Cohorts, One Frozen Instrument
2024 cells (○), the 2026 English arm (◆) and Chinese arm (▲) over the country field - descriptive only. Here colour marks the administration arm, not developer origin (Figures 4 and 6 colour by origin). The cohorts share no models and differ simultaneously in recorded serving conditions (local versus cloud), model scale, retry intensity, language coverage and survivorship; the bootstrap estimators differ too (2024: item, 2026: cluster).
Figure 6. Both cohorts share the frozen coordinate system, but no models. They differ in recorded local versus cloud serving, model scale, language coverage, Chinese translations, retry policies (up to fifteen re-asks in 2024 versus three attempts per pass and up to five passes per 2026 collection run) and selection into the usable sample. Markers show observed-mean positions; I've left the uncertainty regions off for readability. Nothing about cause follows from putting the two years on one plot.
On distance, the 2026 English cells span 1.63 to 3.19 map units from the survey reference with a median of 2.24, against 2.71 for the eight 2024 English cells. The Chinese cells span 2.15 to 3.31 with a median of 2.71. The English distances have a tighter interquartile range (0.11 versus 0.36 in 2024) but a wider full range, and the mean pairwise distance between cells is similar (0.82 versus 0.76).
So a narrower spread of distances from the reference doesn't mean the models are closer to each other. Concentration around a distance is not convergence to a point, and it certainly isn't proof that models are homogenising.
The 2024 separation floors don't survive either. The closest 2026 cell, English minimax-m3, is 1.63 units from the reference; 75 of 109 countries (69%) are closer than it is and its nearest non-Western centroid is 0.81 units away.
I compared every 2026 draw against the exact 2024 floors, regenerated at full precision. Against the point-estimate floor (103 of 109 countries, 1.61005849 units), 27 of 33 cells drop below in at least one draw. Against the draw-level floor (88 of 109, 1.11702539), fifteen do. That comparison uses full-precision artefacts, not the rounded display values, and it's slightly unfair because the 2024 floors come from item bootstraps while the 2026 draws are prompt-cluster bootstraps; the like-for-like item-to-item version gives 21 and four.
The independent audit did that calculation and it's archived with the manuscript. What survives across every resample of every usable cell is weaker: at least 47 of 109 countries closer to the reference, and a distance of at least 0.37 units to the nearest non-Western centroid (a distance this time, unrelated to the 0.37 coordinate floor under the worst-case fills). The quadrant persisted. The stronger separation claims I made in 2024 didn't.
What the Traces Say
The 2026 collection also kept something I lacked in 2024: the models' written reasoning before their final answers, or reasoning traces. Four cells, from gemma4:31b and mistral-large-3:675b, supply no traces. I selected 900 excerpts with equal coverage across the thirty cells that do, rather than sampling every output at random.
Five LLM annotators (gemma4:31b, glm-5.3, gpt-oss:120b, mistral-large-3:675b and nemotron-3-super) each labelled all 900 excerpts using the same fixed coding instructions, one excerpt per call. The earlier analysis lacked a shared codebook, retained labels for individual traces and reliability metrics. I replaced it after a reviewer asked for stronger checks.
By strict majority vote, 55% of excerpts reason towards what a typical, average or moderate answer would be, 5% reason in the first person as the respondent, and 42% invoke AI identity or guidelines. The labels overlap. Across the five raters the shares range 50-65%, 2-17% and 35-48%, with Fleiss' kappa of 0.78, 0.35 and 0.75: the first and third codes have substantial agreement, the persona code only fair, so don't read 5% as a precise measure of how often models "inhabit" a persona. Five separate annotation calls also don't make five LLM raters' errors independent.
The 55% needs care. The first code deliberately covers both "what would most people say" and "I'll pick something middling", so it's wider than population-mode estimation. Some excerpts explicitly discuss typical humans or familiar survey responses; others just want a moderate number. The independent audit found positive labels on excerpts that merely restated the assigned role and then picked arbitrarily, which the codebook says to score zero.
164 of the 900 excerpts hit the 2,000-character cap, and two annotators rated sixty excerpts from their own model. And a trace may be a post-hoc story rather than an account of what happened25. So the traces help interpret the answers without establishing how the models computed them, and high agreement on a broad label is not evidence for a narrow reading of it. The reliability table and the codebook are in Coding the Traces.
The second reviewer put the objection better than I would have: "The reasoning-trace analysis itself suggests that models frequently attempt to estimate the typical human response instead of expressing an internal value position. This makes interpreting coordinates as 'model values' somewhat difficult." I'd let that stand.
Means Versus Midpoints
Three different targets get conflated whenever someone reads "the models are far from the human average", and they're nowhere near each other on this map. The survey reference is the projection of the average observed answer to each question. A population's most common answer can differ from its average, and the midpoint of a scale needn't be either.
Answer every question at its scale midpoint and you land at (0.36, 1.90), more secular than Japan. Answer every question with the single most common human response and you land at (−1.47, −1.47), the opposite corner; the weighted and unweighted modes agree. (A vector of per-question modes needn't be the mode of the joint distribution, so treat that point as illustrative.)
Answer at random, fifty times per question for 100,000 simulated cells (seed 11092026; two distinct ordered goals, exactly five of eleven child qualities), and 95.2% land in the models' quadrant. None of that is a calibrated null for what a model does, and neither constructed profile proves what a model is aiming at.
What it does show is why "the models are far from the average person" can't be read as "the models are bad at guessing a typical person", and why a broad "typicality" trace label can't be read as evidence they're guessing well.
Rupprecht's three possibilities, a midpoint habit, an estimate of typicality, and a genuinely modal answer, are all live here. Li, Li and Qiu find homogenisation in model-generated survey samples26; Taday Morocho et al. find no consistent aggregate benefit from persona conditioning across two models, with effects that vary by item and subgroup, so nothing here assumes every persona intervention behaves the same27.
I tested the midpoint story directly. Across cells, the distance between a model's ten raw item averages and the scale midpoints shows no detected association with secularity (Spearman , raw , corrected in a seven-test exploratory family that also includes midpoint distance against self-expression and five tests on excerpt length). The cell-level tests ignore that cells from the same model aren't independent, and a null here doesn't rule out every model sharing the same scale-induced offset. Definitions and counts are in diag_2026_exploratory_correlations.csv.
I also measured how varied the answers were, using entropy: low entropy means a model repeats a small set of answers. Less varied answers tended to lie further from the scale midpoints. For raw answer strings, the correlation is (, post hoc). For the scores calculated from those strings, it's (). The numbers differ because several answer strings can produce the same score.
Score entropy also correlates with uncertainty in a cell's position: (, 33 cells). Here I measure that uncertainty by the geometric mean of the two coordinate standard deviations from the item bootstrap. These three separate tests describe output concentration and its relation to the map. They don't measure how accurately a model guesses a population's most common answer.
What Changes It
So which thing accounts for how much of the variation in positions? I built 1,700 cell-variant-repeat profiles (all 34 cells, ten variants and five repeats, filling 150 missing entries with variant means and 65 with cell means) and split the variation between the levels.
| Term | PC1′ share of the variation | PC2′ share of the variation |
|---|---|---|
| Model | 21.37% | 22.51% |
| Language within model | 10.66% | 6.45% |
| Prompt variant within cell | 34.68% | 31.17% |
| Repeat within variant | 33.29% | 39.87% |
Model identity accounts for more variation than language on both axes. Prompt variants and repeats within a variant each account for more than either. The language row combines the shared English-Chinese difference with differences specific to individual models. These percentages divide the variation I observed; they don't establish its causes.
Individual answers can vary widely even when I can estimate their average quite precisely. Averaging reduces uncertainty about the cell mean without making its answers any less variable. Measuring Uncertainty explains the distinction and the calculation behind these shares.
Both reviewers wanted to know whether the "average person" cue was doing all the work. I compared the six averaging prefixes with the three bare ones. "Bare" here still means a human-role instruction, such as "a person"; it just leaves out "average" or "typical".
All 32 cells I could estimate from those bare prefixes stayed in the quadrant, with 29 further from the survey reference than under the averaging prefixes. This subset check needs only one parsed answer per item and uses item resampling, so it's weaker than the main analysis.
Then I ran the whole English survey again with no persona prefix at all, just the question, the formatting instruction and the same first-person primer. Thirteen of seventeen cells clear the main eligibility bar. All thirteen stay in the quadrant, and twelve move further out: median distance 3.43 units against 2.21 for the matched persona cells. A looser one-answer threshold admits fifteen cells, fourteen of them further out.
The common corner persisted without the persona prefix, and most usable positions were further out. But I collected the control on 8-11 September, after the main run on 4 August. A hosted model could have changed between those dates, so I can't attribute that extra distance to removing the persona.
Nguyen and Ahmad also found sensitivity to prompt tone, with shifts of up to 2.4 map units on their own reconstruction28. The two maps have different fitted scales, so those distances can't be compared directly with mine.
The traces do tie the framing to what the models say they're doing. In the selected English excerpts the typicality label appears in 211 of 265 averaging-prefix traces (79.6%) against 47 of 139 bare-prefix traces (33.8%); in Chinese, 164 of 289 (56.7%) against 37 of 119 (31.1%). Those are descriptive comparisons on a selected sample, not adjusted causal estimates, and the no-persona traces weren't re-coded. But they show geometry can stay put while the stated reasoning changes.
All of which means the prompt is part of what I'm measuring. Models may be answering an inferred interlocutor; political-bias audits have been shown to capture sycophancy towards the perceived auditor29. My protocol asks for a human persona and then starts the model's reply for it, the control keeps the primer, and chat templates may render that primer differently across models. Neither condition measures what a model does by default.
Tight uncertainty on a pooled mean coexists with wide variation across prompts and answers. Moore et al. examine consistency under their own elicitation choices30, Kovač et al. separate stability of a value structure from stability of a ranking, with model-dependent results31, and Huang separates answer-order and wording effects from an inferred underlying stance32.
None of that makes these coordinates immutable traits, and cultural prompting and steering can move them8 33 34. But a position that can be changed by prompting is not thereby an unimportant measurement. Its scope is the protocol that produced it, and the protocol includes you.
Where It Comes From
Nothing in this design identifies a cause. Training-data composition, post-training preferences, scale habits, how the question gets read, and selective answering could all contribute, and the language-dependent refusals keep selection on that list.
Two patterns are worth stating anyway. On four items (homosexuality, abortion, child autonomy and petition-signing) all seventeen English-elicited models differ from the human averages in the same signed direction. There's little net pull towards high rather than low response codes, which rules out neither item-specific agreement bias nor answer-order effects.
Models from the same origin have more similar patterns across the ten raw answers than pairs from different origins. The release-level permutation test gives . Using nine developer families instead of seventeen releases changes it to 0.1508.
Putting each question on a comparable scale changes that picture. After standardising each item with its fixed survey mean and standard deviation, average correlations are 0.849 within Chinese-origin pairs, 0.703 within Western pairs and 0.678 across origins. Exact permutation tests give for the pooled comparison and 0.0052 for the Chinese-only contrast. On the nine family-mean profiles, those values are 0.0159 and 0.0079.
The enumeration is exact for each label-shuffling scheme, which isn't proof the scheme is right, and family averaging changes both the target and the weighting. Neither calculation identifies a causal origin effect. It coexists with the earlier null on origin-by-language interaction because those are different hypotheses. The bundled item moments make the standardised version reproducible without the private fitted model (conf_2026_family_origin_similarity.csv, conf_2026_origin_similarity_standardised.csv).
Related releases can be far apart. kimi-k2.6 and kimi-k2.7-code differ by 0.18 English units, but version and code specialisation change together. The two DeepSeek flash tags differ by 1.03 units in English and 1.60 in Chinese. Those show that nominally sibling releases can differ a lot; they don't say whether specialisation, pretraining, post-training, architecture or hosting did it, and the three short size ladders in the set are too small and too mixed to support a scale law.
The three 2024 dolphin fine-tunes, built by stripping alignment data out, sit near everything else, but that's an unmatched comparison and doesn't separate pretraining from later training.
Prior work supports the possibilities without settling mine. Opinion-related responses change with post-training35 36 37 38, and preference construction reflects design choices by whoever builds the preference data39 40. Pretraining contributes behavioural priors of its own41 35, and creator-associated differences show up on other moral-assessment instruments42.
Models also copy each other, through distillation, shared synthetic data and imitation43 44 45 46 47, and this design doesn't trace training lineage or separate vendor, version and national origin, so the sibling contrasts can't establish a causal hierarchy either.
Scenario-based elicitation and latent steering33 34, culturally grounded personas48, word associations49 and multi-agent debate50 all get at cultural behaviour beyond fixed survey answers. To attribute a change in survey position to a training intervention, I'd need to compare matched models before and after that intervention, or compare a teacher model with its distilled student. I haven't run those experiments on these models and items.
What This Can't Tell You
A survey position needn't predict behaviour in an open-ended conversation or reveal a held value. Some questions also assume the respondent has a country, which my country-free personas leave unspecified. The trace label combining typicality with moderation can't resolve those ambiguities.
Public survey wording may be in the training data, and some excerpts name WVS responses. That fits several possible answering strategies. It doesn't by itself prove memorisation or accurate recall of population statistics.
The human comparison pools historical survey responses. It represents neither human values in 2026 nor the world's population. Fitting the Map and Missing Answers explain which answers I reconstructed and which I estimated. Those repairs retain more respondents, but don't remove uncertainty or establish agreement with published WVS country scores.
Country and model positions have different sources of error. The ellipses leave out uncertainty from human-survey sampling, estimates of missing answers, fitting the map and changes to hosted models.
Ten prompt variants give few clusters, and I haven't measured the actual coverage of the "95%" ellipses. In the worst cell, fifty calls carry about as much information as ten independent answers.
Fitting the axes also requires choosing a rotation, which aims to make each question contribute mainly to one axis. I checked six conventions. The stored rotation sits about 1.5° from its own criterion's optimum, and two alternatives put three or four cell means outside the quadrant. The headline therefore depends on the chosen rotation; Rotation Sensitivity gives the comparison.
The tested models aren't a random sample of open-weight models, related releases may share training histories, non-detection isn't equivalence, and four items or two coordinates don't identify a complete cultural profile. And because prompt language, translation and inferred context all change together, none of this can test linguistic relativity, whatever my 2024 post implied about Sapir-Whorf; Au's counterfactual-language experiments are the standing caution here51.
My aim was narrower: make one instrument-based measurement reproducible and say plainly what it can't support.
Two more things, because region names invite them. "Protestant Europe" and "Confucian" are the map's analytical categories, not verdicts on societies, and proximity on two axes doesn't describe a culture, establish that a model represents any population, or say which values are correct. Steering a model towards any cultural profile is a normative choice this study doesn't make. And model-written explanations, coded by other models, aren't evidence of anything consciously or internally held.
If you're deploying one of these models somewhere culture matters, test how it responds to different prompts and how it behaves in your actual setting. Calling a model "Western" doesn't tell you how it will behave there. Two coordinates establish neither cultural neutrality, a universal Western population estimate, nor fixed internal values.
Still Open
Does changing the survey language activate different patterns inside a model, or change how it uses the same patterns? Interpretability researchers call some identifiable patterns features, and study how they interact in circuits52 53 54. Tools now target cultural and value-related features specifically55 56 57 58 59. There's also evidence for concepts represented independently of language60, and for steering output language separately61.
I haven't applied those methods to this survey, so this study doesn't establish a mechanism for the language shifts. The tools have limits too. Steering can move several value dimensions at once34, and sparse-autoencoder steering doesn't consistently outperform prompting or fine-tuning62. Features can also absorb one another or leave parts of the model's behaviour unexplained63 64 65. The 2,000-character verbal traces can't resolve those internal mechanisms.
The blinded 150-excerpt worksheet for human coding exists, with five examples per trace-emitting cell and the LLM labels withheld, and no human has coded it. The matched training experiment from Where It Comes From, base model against post-trained or teacher against student, hasn't been run either.
There's a smaller experiment to run first. One of the ten questions asks which qualities children should learn at home, and my Chinese wording can be read as asking about studying at home. Eleven of the fifteen models that show their reasoning used study-related wording, including three Western models.
The average self-expression shift survives the exclusions I tried, while the evidence for the autonomy-item shift weakens. I can't show you what the models would have said to a better translation, because I haven't asked. The next check is to ask them.
The rest is reference material for readers who want to inspect or reproduce the study. Start with the models, questions, map calculation, or uncertainty checks.
The Models Tested
The eleven 2024 cells are in the table under Four Things Wrong. My records describe local Ollama serving with Q4-quantised community GGUF builds; I didn't keep immutable per-run weight snapshots or hardware logs.
Five recorded 2024 entries never produced a usable corpus: yi:34b, aquilachat2:34b, glm4:9b, xuanyuan:70b and kingzeus/llama-3-chinese-8b-instruct-v3. My notes from the time describe punctuation-only answers, prompt echoing, unintelligible output and intermittent failure, and since I didn't keep the raw failed outputs those descriptions can't be independently rechecked. The hand-written Modelfiles also weaken the per-name attribution: the yi and glm wrappers both reference the AquilaChat2 artefact. So the list doesn't establish five distinct failed models, and the 4 of 9 historical success fraction is presented alongside 4 of 7 as a denominator sensitivity, not as a reliably identified rate.
The 2026 set is ten Chinese-origin models and seven Western ones, all served through cloud endpoints. Serving precision wasn't retained in the collection records, and the recorded model tags and call metadata don't guarantee immutable weights or that a future hosted call would reproduce them.
Positions in the table are direct projections of observed per-item means; the standard deviations come from prompt-cluster resampling. The language displacement is the plug-in norm, with percentiles of replicate norms as its interval. The underlying artefacts are llm_parse_rates_2026.csv, llm_ellipses_2026.csv and llm_language_effects_plugin_2026.csv.
| Model | Origin | Parse en | Parse zh | en position (PC1′, PC2′) | zh position | ‖δₘ‖ [95% CI] |
|---|---|---|---|---|---|---|
deepseek-v4-flash | Chinese | 100.00% | 100.00% | (1.31 ± 0.15, 1.12 ± 0.09) | (1.34 ± 0.18, 1.66 ± 0.10) | 0.54 [0.34, 0.89] |
deepseek-v4-flash:0731 | Chinese | 100.00% | 100.00% | (2.15 ± 0.22, 0.52 ± 0.16) | (2.76 ± 0.18, 0.93 ± 0.08) | 0.74 [0.37, 1.24] |
deepseek-v4-pro | Chinese | 100.00% | 100.00% | (1.00 ± 0.16, 1.92 ± 0.24) | (1.36 ± 0.08, 2.14 ± 0.13) | 0.42 [0.12, 0.94] |
glm-5.1 | Chinese | 100.00% | 100.00% | (1.46 ± 0.14, 1.57 ± 0.18) | (1.96 ± 0.18, 1.89 ± 0.22) | 0.59 [0.16, 1.20] |
glm-5.2 | Chinese | 99.80% | 100.00% | (2.24 ± 0.31, 0.51 ± 0.14) | (2.51 ± 0.24, 1.35 ± 0.18) | 0.88 [0.50, 1.50] |
kimi-k2.6 | Chinese | 100.00% | 100.00% | (1.43 ± 0.20, 1.63 ± 0.12) | (2.22 ± 0.17, 1.81 ± 0.10) | 0.81 [0.30, 1.36] |
kimi-k2.7-code | Chinese | 100.00% | 100.00% | (1.56 ± 0.19, 1.51 ± 0.13) | (2.09 ± 0.23, 1.85 ± 0.13) | 0.64 [0.12, 1.27] |
minimax-m2.7 | Chinese | 100.00% | 99.80% | (1.59 ± 0.12, 1.19 ± 0.07) | (1.94 ± 0.07, 1.17 ± 0.07) | 0.35 [0.11, 0.63] |
minimax-m3 | Chinese | 100.00% | 96.20% | (0.73 ± 0.15, 1.38 ± 0.13) | (1.37 ± 0.10, 1.58 ± 0.10) | 0.67 [0.38, 1.01] |
qwen3.5:397b | Chinese | 99.40% | 99.80% | (1.44 ± 0.10, 1.64 ± 0.12) | (2.04 ± 0.10, 1.48 ± 0.08) | 0.62 [0.43, 0.88] |
gemma4:31b | Western | 94.40% | 99.20% | (1.91 ± 0.10, 1.94 ± 0.07) | (2.34 ± 0.22, 1.46 ± 0.14) | 0.64 [0.21, 1.20] |
gpt-oss:20b | Western | 100.00% | 99.80% | (2.08 ± 0.08, 1.41 ± 0.10) | (1.43 ± 0.10, 1.85 ± 0.10) | 0.78 [0.56, 1.03] |
gpt-oss:120b | Western | 98.00% | 97.00% | (2.47 ± 0.11, 1.96 ± 0.06) | (2.78 ± 0.08, 1.76 ± 0.08) | 0.36 [0.16, 0.62] |
mistral-large-3:675b | Western | 100.00% | 100.00% | (0.72 ± 0.22, 2.11 ± 0.07) | (2.20 ± 0.19, 1.98 ± 0.09) | 1.49 [0.92, 2.00] |
nemotron-3-nano:30b | Western | 99.80% | 99.60% | (1.60 ± 0.09, 1.56 ± 0.09) | (1.72 ± 0.12, 1.66 ± 0.14) | 0.16 [0.06, 0.51] |
nemotron-3-super | Western | 100.00% | 99.80% | (1.49 ± 0.25, 1.49 ± 0.07) | (1.92 ± 0.11, 1.77 ± 0.10) | 0.52 [0.13, 1.04] |
nemotron-3-ultra | Western | 92.00% | 82.40% | (1.91 ± 0.19, 1.43 ± 0.10) | excluded | not estimable |
The Chinese nemotron-3-ultra cell parses only 4 of 50 abortion trials, fails the ten-response rule, and has no estimable displacement. Every other cell passes. Thresholds of five and 25 parsed answers give the same inclusion decisions in the persona corpus, which doesn't imply the later no-persona controls pass those thresholds.
One implementation note for anyone rerunning the cluster bootstrap: missing variant-item groups must not be allowed to become non-finite sums that silently invoke the pooled fallback. The zero-fill rule is under Measuring Uncertainty; regression tests cover this sparse-cell case, and the final figures and tables come from the corrected path.
The Ten Questions
The items, with their survey codes and native ranges: A008 happiness (1-4), A165 trust (1-2), E018 respect for authority (1-3), E025 petition (1-3), F063 importance of God (1-10), F118 justifiability of homosexuality (1-10), F120 justifiability of abortion (1-10), G006 national pride (1-4), Y002 post-materialism (two ranked goals from four), and Y003 autonomy (up to five child qualities from eleven).
The official Y003 transform is A029 + A039 − A040 − A042: independence and determination count positively, religious faith and obedience negatively. The released app/cloud_survey.py has the complete English and Chinese item messages, the format instructions, the prefix variants and the system primer.
All persona prefixes sit in the user turn. Prefixes 0, 1, 3, 4, 6 and 7 contain "average" or "typical"; prefixes 2, 5 and 8 say "a human being", "a person" or "an individual"; prefix 9 says "a world citizen". The separate system message contains "Sure thing! Here is my numerical answer:" and, in Chinese, 好的！这是我的数字答案： (hǎo de! zhè shì wǒ de shùzì dá'àn:). The no-persona control keeps this first-person prefill. The nemotron-3-ultra refusal quoted under Who Refuses is the raw terminal response for prefix 0, repeat 0, with the full sentence preserved.
The 2024 Chinese wording of the homosexuality item mislabelled both poles "always justifiable"; the 2026 version labels the low pole "never justifiable". Some earlier Chinese prefix translations had collapsed into duplicates and were revised. The format instructions and the primer were also translated for 2026, whereas the 2024 Chinese collection mixed Chinese questions with English formatting text.
No answer-token likelihoods were retained, so this collection never compared likelihood-based scoring with generated answers. These corrections complicate cross-year comparison, and they don't guarantee semantic equivalence with the official WVS Chinese questionnaires.
Anchor order differs for abortion: English names 10 before 1, Chinese names 1 before 10; homosexuality names 1 first in both. And the Chinese averaging-family prefixes 0, 3 and 6 use 普通 (pǔtōng, "ordinary"), which needn't denote a statistical average.
Fitting the Map
The paper has the full derivation; this is the outline. I merged the EVS Trend File 1981-201710 and the WVS Trend File 1981-202211 with the published syntax, kept respondents surveyed from 2005 onwards (variable S020), and recoded out-of-range user-missing sentinels. Missing Y003 is reconstructed as A029 + A039 − A040 − A042 only when all four child-quality indicators are valid 0/1 responses, with valid delivered indices preserved.
I then required at least six observed items, counting that exact reconstruction as observed rather than imputed. The scoring rule agrees with all 267,160 directly supplied post-2005 WVS indices that have complete constituents. In the fitted sample Y003 has 265,773 direct, 117,075 reconstructed and 9,534 missing values, and recovering it before filtering admits 752 additional respondents.
The fitted sample is 392,382 respondents across 112 country codes. The map carries 389,341 respondents in 109 countries and territories; three codes absent from the country-code table are excluded after fitting. Country aggregation applies the original national weight S017, so each country pools its retained 2005+ records with survey years contributing according to their retained weight mass rather than equally. Nothing is weighted to represent the world's population.
Probabilistic principal component analysis66 fits an unweighted two-dimensional Gaussian latent model, holding the observed-item means and standard deviations fixed. The fit optimises the observed-data likelihood directly with L-BFGS-B, integrating over missing entries, and only then completes them with conditional means.
The code retains the projection interface from pca-magic67 (Apache-2.0) but replaces its inherited approximate missing-data loop after the final implementation audit. That loop fed imputed values back into the residual-variance update and omitted the conditional second moments required by the probabilistic PCA model. The replacement passed independent likelihood checks; The Wrong Coordinates explains how this affected the analysis.
On standardised observations the model covariance is , and and the positive residual variance are retained with the fit. One spectral start and two seeded perturbations are optimised; all three must reach a maximum absolute gradient of the negative log-likelihood per informative row of at most within 1,000 iterations, or fitting raises, and the highest-likelihood converged start is kept. Multiple starts don't guarantee a global optimum. Completion uses the conditional mean for each observed-item pattern.
Conditional-mean imputation uses cross-item information, unlike marginal-mean filling, though it still attenuates variability; complete-case analysis would instead select heavily on which questions each country-wave happened to field. The retained scores explain 41.3% of the variance of the completed matrix, not 41.3% of ten unit-variance observed columns.
Each raw item is standardised with its fitted mean and standard deviation, and those parameters are frozen. The native scales run from 1-2 to 1-10, so applying axes fitted to standardised inputs directly to raw answers gives item ranges and means an influence they shouldn't have, which is the 2024 bug.
For the rotation, let be the covariance of the completed standardised matrix. I orthonormalise the fitted Gaussian loading subspace, then diagonalise score covariance within it to get the projection axes and the diagonal score covariance , with and . is distinct from the Gaussian loadings; these identities don't require to be eigenvectors of the full completed covariance or to reconstruct its ten-dimensional variation.
A varimax rotation is fitted once to the score matrix and stored. Its stored angle, −37.9°, is the solver's output at tolerance on a nearly flat criterion whose optimum lies about 1.5° away. The score-side criterion with Kaiser row-normalisation differs from textbook loadings rotation and from parts of Rohe and Zeng's procedure, which declines Kaiser normalisation68.
The rotated score covariance is ; an orthogonal rotation preserves geometric orthogonality of the basis, not uncorrelated scores when the retained eigenvalues differ, which is why the plotted axes correlate at . Orientation is fixed by requiring F118 to have a positive self-expression coefficient and F063 a negative secular-rational one.
Then the rescaling. The WVS affine constants are and . Rotated scores are first divided by their fitted standard deviations, , to reach the unit-variance scale those constants presuppose. For any respondent, country aggregate or model response profile,
Here are ten-vectors, is , and the operations involving , and are elementwise. The rescaled axes are PC1′ (survival to self-expression) and PC2′ (traditional to secular-rational). Every parameter is frozen at survey fitting, and countries and models go through the same project() method. Correcting translates points and fitted references together, which preserves their relative geometry without automatically preserving thresholds fixed at zero.
Three validation checks address different failure modes. Path identity: projecting 275,955 complete-case survey rows through the public API reproduces their internally fitted coordinates with zero maximum discrepancy at machine precision, which guards against inconsistent transformations while leaving open a conceptual error both paths share.
Correction accounting: per-axis regressions against the 2024 published coordinates have and with slopes 1.34 and 1.29, describing the correction rather than agreement with published WVS scores or rank preservation.
Coefficient orientation: the entries of give each standardised item's contribution to each rotated score, which is a face-validity check and not external calibration; these coefficients differ from Gaussian loadings and from item-score correlations, and reverse-keyed items have to be read with their coding.
A separate 20-seed likelihood refit finds a rotation span of about 0.000016° and a maximum country-coordinate range below map units. That bounds initialisation with the rotation tolerance fixed; it says nothing about the varimax stopping error, or about survey, missingness-model or rotation-choice uncertainty.
Missing Answers
Some values are missing because a question wasn't fielded in a particular country-wave. An absent derived Y003 column isn't evidence that the four recorded constituent answers are unavailable, which is why reconstruction comes first. Model-based imputation retains the 116,427 rows that a complete-case analysis would discard, though missingness by design alone doesn't establish the assumptions needed for unbiased imputation69.
After recovery no fitted entity is wholly missing Y003, but other whole-country item gaps still need extrapolation from observed relationships. Wholly unfielded country-waves concentrate on the homosexuality item, which has the largest positive self-expression coefficient; Egypt, Kuwait and Tajikistan have no observed responses to it at all. National-pride gaps partly reflect that the question doesn't apply to non-nationals.
The independent audit asked what happens if you fill every wholly unfielded country-item cell with the worst-case value. The 2024 minimum stays at 103 of 109 countries closer. The 2026 minimum drops from 75 of 109 to 73. Both non-Western centroid minima don't move. The bundled aggregate missingness report separates observed from imputed ingredients.
As a check on the reconstruction, I hid the Y003 answers from countries that had them and asked the frozen fit to guess them back. The median absolute country-mean error was 0.236 index units and the maximum 0.863, which through that item alone is 0.092 and 0.337 map units. That's an in-sample diagnostic, not held-out validation and not an error estimate for the entries that really are missing.
Among the 172,503 remaining imputed entries across all items, 2,758 fall outside their response ranges; clipping them diagnostically shifts country means by at most 0.0143 map units, which doesn't make clipping the correct estimator.
Measuring Uncertainty
Because the projection is affine, a cell's point estimate comes straight from its observed per-item means, and arbitrarily pairing answers into pseudo-respondents wouldn't change it. For either scalar coordinate,
Shared prompt variants can induce covariance between items, and the affine identity doesn't make those terms vanish.
The table in What Changes It splits the total variation of the constructed profiles using a nested sums-of-squares decomposition. Those parts give additive shares of the same total. Normalised mean squares divide by different degrees of freedom, so they wouldn't give additive shares and aren't substituted here. Variation across profiles is distinct from uncertainty in their pooled mean.
Two resampling procedures. The item bootstrap () resamples each item's responses independently and projects the resulting means; it omits cross-item covariance and gives no guaranteed lower bound.
The cluster bootstrap () resamples the ten prompt variants, carrying all their items and repeats together, and is primary whenever variant identifiers exist. Missing variant-item groups contribute zero counts and sums (the implementation zero-fills them before summing sampled counts and answers), and the pooled-item fallback applies only when a sampled replicate has no valid response at all for an item. The 2024 corpus permits only item resampling; the 2026 design supports both. Ten deliberately chosen variants aren't a random sample of prompts and inference with ten clusters is approximate12.
The intraclass correlation says how alike the five repeats under one variant are, and high values mean repeating the wording buys little: on the completed 2026 corpus the median per-item ICC is 0.10, 51 of 340 cell-item pairs (34 cells by ten items) exceed 0.5, and the maximum is 1.00. With five repeats per variant those imply design effects up to 5.0 (1.4 at the median), so at the extreme fifty calls carry the information of roughly ten independent draws, one per prompt variant. Neither bootstrap propagates imputation, model-snapshot, survey-design or all elicitation uncertainty.
For the language contrast, model 's displacement is , the difference of observed run means, and is its plug-in norm. Independent resampling of the two runs targets separately estimated means; joint resampling of matched variants would target a different dependence structure and isn't used in the primary contrast.
Per-model norm intervals are percentiles of replicate norms of the same estimator. Their positive lower endpoints are not evidence of a nonzero vector, because a norm is nonnegative and folds noise away from zero. The mean of replicate norms is a separate, generally larger summary and not the point estimate. Signed components support directional inference. Across-model intervals resample the sixteen estimated vectors with their within-model estimates held fixed, so they describe dispersion in this tested set and omit within-model uncertainty.
Weighting the nine developer families equally gives a mean magnitude of 0.69 [0.53, 0.92], a mean PC1′ change of +0.50 [0.25, 0.79] and a mean PC2′ change of +0.10 [−0.10, 0.28]. This averages individual norms within families, rather than taking norms of family-mean vectors, and bootstraps the family summaries.
Paired resampling of the same variant identifiers in both languages leaves point contrasts unchanged, with per-component standard deviations 0.40 to 1.16 times the independent-run ones. The two procedures target different repeated-prefix designs, and neither is a random sample of all prompts.
The plotted regions use the bootstrap covariance :
They concern a cell's mean position, not the spread of individual responses. The radius is a conventional nominal 95% choice. Approximate ellipticity of the bootstrap clouds is a shape diagnostic, not a coverage test. A Gaussian ten-cluster benchmark illustrates coverage near 85% without estimating actual coverage here, and a Hotelling-style small-sample radius, adjusted for the bootstrap covariance normalisation, is more conservative under that benchmark's assumptions. Actual coverage is unknown.
The survey reference is the projection of the unweighted observed-item marginal means, (0.038, −0.10), equal to the affine offset by construction. It differs from the mean of conditionally completed respondent profiles, about (0.016, −0.104), nor is it a world-population-weighted cultural centre. The quadrant lies above it on both axes.
Per replicate I report distance to it, the percentage of the 109 mapped countries closer to it, and distance to the nearest non-Western centroid, where non-Western excludes Protestant Europe, English-Speaking and Catholic Europe as an operational grouping. "Every generated replicate" is a finite descriptive statement that neither enumerates the bootstrap distribution's support nor provides simultaneous coverage, and no binomial tail or union bound turns zero observed crossings into a guarantee.
Four alternative references, using completed scores for all fitted or only mapped respondents, each unweighted or S017-weighted, leave all 44 point estimates and all generated replicates in the quadrant, change point distances by less than 0.037 map units and country shares by at most 2 of 109 (1.83 percentage points); that checks the mean-reference choice and nothing wider (validation_reference_sensitivity.csv).
The worst-case bounds under Who Refuses assign every unobserved terminal answer each admissible extreme in turn and project the resulting profiles23. They cover terminal failures under the implemented retry protocol, not unrecorded rejected drafts or prompts that were never run.
Rotation Sensitivity
The rotation grid reports counter-clockwise angles for the same fitted components.
| Criterion | Angle | Means inside | Draws inside |
|---|---|---|---|
| Scores, Kaiser (used) | −37.9° | 33/33 | 330,000/330,000 |
| Scores, no Kaiser | −4.1° | 29/33 | 281,627/330,000 |
| Whitened scores, Kaiser | −38.7° | 33/33 | 330,000/330,000 |
| Basis , Kaiser | −32.3° | 33/33 | 329,890/330,000 |
| Basis , Kaiser | −27.0° | 33/33 | 327,676/330,000 |
| Basis , no Kaiser | −10.0° | 30/33 | 297,915/330,000 |
The comparison holds the fitted subspace and the model responses fixed, reapplies the same sign and order anchors, and recomputes each rotation's score standard deviations before applying the same reference boundary. Alternative criteria change the operational meaning of the axes, and none is automatically an equally valid Inglehart-Welzel measurement; the counts concern eligible point means and finite generated draws.
I use unwhitened empirical scores and haven't established the independent heavy-tailed latent-factor assumptions of Rohe and Zeng68 for these ordinal inputs.
The chosen and whitened-score criteria differ by 0.8° at the released tolerance of , which the comparison pins for both. The independent audit's scan at tighter solver tolerance converges to −39.4° for the primary criterion and −40.1° for whitened scores, narrowing the difference to 0.7°.
At that tighter optimum the rotated-score standard deviations are (1.46, 1.36). Country coordinates move by at most 0.061 map units and model coordinates by at most 0.060; model reference distances change by at most 0.023. All 33 means and all 330,000 draws stay in the quadrant. That 1.5° stopping-point sensitivity is separate from likelihood-fit seed stability, and the released tables keep the original tolerance.
Rohe and Zeng explicitly decline Kaiser normalisation, so agreement with the general score-side approach doesn't establish their assumptions here, and agreement between two choices doesn't make a rotation uniquely correct. The primary and whitened-score criteria keep all 33 means and 330,000 draws inside; both Kaiser-normalised basis alternatives keep every mean but let some draws out; the two unnormalised alternatives keep only 29 or 30 means.
The recovered Y003 rotated projection coefficient is (0.20, 0.31), and its correlation with importance of God is −0.38 across the 371,591 retained respondents with both items observed, excluding imputed entries.
Six further diagnostics live here too. Inclusion: persona-cell eligibility is unchanged at thresholds of five, ten and 25, separately from the no-persona sensitivity.
Region shape: empirical Mahalanobis-distance quantiles diagnose the bootstrap-cloud shape, but swapping a normal-theory radius for an empirical radial quantile isn't an independent check of coverage. Justifiability-item neutralisation: replacing every homosexuality and abortion answer with its human item mean moves three cells to or beyond the self-expression reference, Chinese deepseek-v4-pro, English minimax-m3 and English mistral-large-3:675b, all of which stay above the secular reference; that's a counterfactual on every answer to two items, not the terminal-failure bound.
Item influence: leave-one-item-out projection across the 33 cells gives the largest mean displacement to homosexuality (0.91 units), then child autonomy (0.56) and abortion (0.32), values that depend on the instrument and the neutralisation rule.
Region assignment: the classifier and centroid rules agree on 27 of 33 cells, and their disagreement is a reason to qualify regional names rather than to pick a classifier. Release and scale comparisons: the sibling releases and the three short size ladders are descriptive and hold no other training or serving feature fixed.
Prompt Checks
These checks exist because two anonymous reviewers asked for them. They're post-confirmatory, and they don't by themselves test whether framing causes survey-statistics reasoning.
The six averaging prefixes contribute 300 planned trials per cell, the three bare prefixes 150 and the world-citizen prefix fifty. Each subset needs only one parsed response per item, explicitly weaker than the main ten-response rule, and both sides of a subset contrast use item resampling conditional on the selected prefixes, which omits cross-item covariance. Cluster resampling degenerates when there's only one prefix. It stays well defined with fewer than ten, just weaker.
| Prefix family | cells | estimable | in quadrant | median distance | min PC1′ | min PC2′ |
|---|---|---|---|---|---|---|
| All ten (protocol) | 33 | 33 | 33 | 2.40 | 0.72 | 0.51 |
| Averaging cue (6) | 33 | 33 | 33 | 2.31 | 0.31 | 0.57 |
| Bare (3) | 33 | 32 | 32 | 2.74 | 0.91 | 0.06 |
| World citizen (1) | 33 | 33 | 33 | 3.27 | 1.21 | 0.93 |
All 32 estimable bare-prefix cells remain in the quadrant; English gemma4:31b isn't estimable because that subset has no parsed abortion answer. Relative to the averaging prefixes, 29 of 32 lie further from the survey reference and 31 of 32 are higher on self-expression. The matched median reference distances are 2.74 versus 2.27 units, with a median self-expression change of +0.61, and 20 of the item-bootstrap self-expression intervals exclude zero. The table's averaging median of 2.31 uses all 33 eligible averaging cells, hence the different number. These subset comparisons needn't describe each of the ten variants separately.
The no-persona condition is English-only. It removes the persona prefix and keeps the item, the formatting instruction and the trailing first-person system primer, so a system message is still in play. Of 8,500 scheduled trials, 8,442 unique terminal records contain 13,523 recorded attempts; qwen3.5:397b has 450 records and nemotron-3-ultra 492. Missing scheduled trials, empty responses and explicit refusals are reported separately in the coverage diagnostics.
Under the main ten-response rule, thirteen cells are eligible, all thirteen stay in the quadrant and twelve are further from the reference, with median distances of 3.43 without the prefix against 2.21 in the matched persona cells. A separately labelled threshold-one sensitivity includes fifteen cells, fourteen of them further, and isn't the primary result.
The persona-run side keeps its prompt-cluster replicates while the no-persona side uses item resampling, since there's only one prefix; I combine 2,000 independent draws per side, with independent child random-number streams to avoid accidental covariance. Eleven of thirteen primary secular-axis intervals are wholly positive (twelve of fifteen under the relaxed rule). The number of generated replicates isn't the number of independent trials or clusters.
The failures are worth reading individually. gemma4:31b has 301 terminal parse failures across six wholly unparsed items, but not 301 established refusals: some of those responses contain substantive values in a format the parser rejects. qwen3.5:397b's fifty absent abortion trials aren't refusals, and its national-pride item yields seven parsed answers out of fifty recorded trials, the rest being empty or otherwise unparsed. gpt-oss:120b has 167 of 500 failures and nemotron-3-ultra 82 of 492.
Those differences show that protocol and eligibility matter without identifying why any model failed. The controls were collected on 8-11 September against the main collection's 4 August, and cloud tags, especially the undated DeepSeek ones, may resolve differently across those dates, so the geometric differences combine possible prompt, hosting and selection changes.
The control is also incomplete for some scheduled trials, its one-response analysis is sensitivity only, and the two sides use different resampling. The prefix-conditional trace rates under What Changes It show geometry can hold still while verbal strategy changes.
Coding the Traces
The submitted version of the paper coded the traces with eight parallel agentic LLM sessions grouped by vendor, with differing operational definitions and no retained trace-level labels. The final published version replaced those counts with a frozen-codebook, trace-level majority vote. "Majority vote" doesn't mean expert adjudication.
Five annotators, gemma4:31b, glm-5.3, gpt-oss:120b, mistral-large-3:675b and nemotron-3-super, each coded all 900 selected excerpts. Each call supplies the frozen codebook, the administration language, the prefix, the item wording, the stored excerpt and the terminal answer, at temperature zero, without the earlier coding. That gives independent calls, not independent error processes, and not blindness to model-identifying text. The codebook's machine key for the first category covers typicality or moderation, so literal population-mode estimation is narrower than the coded construct. Three or more positive votes assign a label, and the categories can overlap.
| Metric | Typicality/moderation | Persona reasoning | AI identity/guidelines |
|---|---|---|---|
| Majority-vote share | 55% | 5% | 42% |
| Per-annotator range | 50-65% | 2-17% | 35-48% |
| Fleiss' κ | 0.78 | 0.35 | 0.75 |
| Krippendorff's α | 0.78 | 0.35 | 0.75 |
| Pairwise Cohen's κ range | 0.70-0.86 | 0.17-0.63 | 0.71-0.79 |
The first and third codes show substantial agreement under conventional descriptive bands; persona reasoning has fair agreement, with raters differing over first-person phrasing embedded in a third-person simulation. High agreement on the broad first code doesn't establish validity of a narrower "population mode" reading. The independent audit found positive labels on role restatements followed by arbitrary picks, which the codebook directs to zero; that's a coding error rather than intended category scope. Those checks are archived with the manuscript and aren't a completed blinded human annotation study.
Censoring and self-rating: 164 excerpts sit at the 2,000-character cap, including 58 of 60 from qwen3.5:397b and 69 of the 177 mixed-script excerpts. gpt-oss:120b and nemotron-3-super each judged sixty own-model excerpts and are the highest persona raters (16.6% and 15.6%).
Removing only the matching model's own rating on each affected excerpt gives 487 of 891 typicality/moderation, 41 of 899 persona and 379 of 887 guideline labels among strict-majority decisions, with two-two ties of nine, one and thirteen. Counting every tie first negative and then positive gives full-sample bounds of 54.1-55.1%, 4.6-4.7% and 42.1-43.6%. None of that establishes independent rater errors or human validity.
Script-based language labels reproduce 554 English, 177 mixed and 169 Chinese excerpts; they're mechanical classifications by CJK-character proportion, not observations of a model's internal reasoning language. A blinded 150-excerpt worksheet is prepared for optional human coding, five per trace-emitting cell with the LLM labels withheld. It isn't completed human validation and no human-LLM agreement is claimed. Reliability here is the multiple-annotator alternative the reviewers asked for; human coding could strengthen construct validation and isn't represented as done.
The coding criteria, with formatting adapted (section headings, numbering and the example output line omitted; the complete frozen prompt is in the released code, and the original machine keys are kept):
You are coding a reasoning trace produced by a language model that was asked to answer one item of a values survey. Read the trace and assign three binary codes. Judge only what the trace says, not the final answer's correctness.
modal_targeting (1/0): the trace uses what a typical, average, common, majority or "most people" answer would be, or population statistics, survey distributions, or an explicit aim at a "moderate" / "middle-of-the-road" answer, as the criterion for choosing the answer. Count it even if the final answer then deviates from that typical answer. Do not count: merely restating the assigned role ("I am an average person") without using typicality to choose; picking the scale midpoint only as a hedge ("I'll say 5, the middle") with no claim about what people commonly answer; AI-neutrality on its own.
persona_reasoning (1/0): the trace deliberates in the first person about the values, beliefs, experiences or circumstances themselves, as the respondent ("I'm fairly happy with my life", "God matters a great deal to me", "I have signed petitions before"), and that first-person position drives the answer. Do not count third-person simulation ("an average person would probably say"), or reasoning about how to simulate a person.
guideline_citation (1/0): the trace explicitly refers to policies, guidelines, safety, disallowed or sensitive content, or runs a harm/risk/compliance check; or it invokes the model's identity as an AI as a limitation or constraint on answering ("as an AI I don't have personal beliefs / a nationality / can't take a stance"). Do not count a bare "As an AI, I'll simulate an average person" that carries no limitation or constraint.
The codes are independent: any combination of 0s and 1s is allowed. Reply with exactly one JSON object and nothing else.
Reproducing This
The corrected code is released under the v1.1.0 tag, dated 14 September 2026. Two archives attached to that release supply the retained model responses (69 files, fixed by a SHA-256 manifest in docs/reproduction-data.json) and the aggregate artefacts cited in the paper: model-cultural-comp-responses-2026-09-12.tar.gz and model-cultural-comp-paper-results-2026-09-12.tar.gz. The repository's docs/REPRODUCING.md documents their contents, installation and the provenance of the licensed inputs.
The licensed survey data and the fitted data/cultural_map_model.npz instrument aren't redistributed; the coordinate stages need them, but the supplement supplies the item moments, rotated projection coefficients and rotated-score standard deviations needed to reconstruct the published cell coordinates.
The additional diagnostics from the independent audit (tighter rotation tolerances, broader Y003 exclusions, human-sample baselines and like-for-like bootstrap comparisons) are archived with the manuscript sources, separately from the immutable release supplement. The paper figures use the released numerical outputs through a manuscript-supplied paper/research_tools/render_paper_figures.py adapter for fonts, canvas sizes, colours and legends.
The dated analysis plan and its deviations ledger are docs/analysis-plan-2026.md. Where I've said "pre-specified" I mean that document, and it first entered version control in v1.1.0 with author-recorded dates, so the repository can't independently establish that the plan preceded the analyses. The repository supplies survey download and merge instructions rather than protected microdata.
Data-free regression tests run through pytest, and the README gives separate commands for instrument validation, model analyses, diagnostics and figure generation. A passing test suite doesn't by itself reproduce every published number; reproduction also needs the recorded inputs, a compatible environment, the analysis commands and inspection of their outputs.
Everything here measures reported model survey positions with publicly documented instruments. The survey data come from the World Values Survey Association, the European Values Study and GESIS, used under their data-use agreements, and anyone reproducing the coordinate stages has to obtain the IVS microdata separately under the same agreements. No new human participants were recruited. The item texts are © WVS/EVS and are reproduced here for research and replication. The pipeline retains interface code adapted from pca-magic (Apache-2.0) with its fitting loop replaced. I used Claude and Codex for coding, analysis checks, reference verification and drafting, and reviewed the resulting work.
Footnotes
-
Shav Vimalendiran (2024). Cultural Bias in LLMs. Blog post, shav.dev, 20 July 2024. Superseded and corrected by this paper. ↩
-
David Haslett, Linus Ta-Lun Huang, Leila Khalatbari, Janet Hui-wen Hsiao, and Antoni B. Chan (2025). Made-in China, Thinking in America: U.S. Values Persist in Chinese LLMs. arXiv preprint. arXiv:2512.13723. ↩
-
Leon Yin, Davey Alba and Leonardo Nicoletti (2024). OpenAI's GPT Is a Recruiter's Dream Tool. Tests Show There's Racial Bias. Bloomberg, 7 March 2024. Experiment code and methods. ↩
-
OpenAI (2026). How ChatGPT adoption has expanded. OpenAI Signals, 30 June 2026. Covers individual ChatGPT plans; primary language is based on the plurality of a user's messages, not necessarily a majority. ↩
-
Ruth Appel, Peter McCrory, Alex Tamkin, Miles McCain, Tyler Neylon, and Michael Stern (2025). Uneven Geographic and Enterprise AI Adoption. Anthropic Economic Index, September 2025 report; arXiv version, submitted November 2025. Geographic analysis uses Claude.ai Free/Pro conversations from 4-11 August 2025, divided by working-age population. ↩
-
Ronald Inglehart and Christian Welzel (2005). Modernization, Cultural Change, and Democracy: The Human Development Sequence. Cambridge University Press. ↩
-
Arnav Arora, Lucie-Aimée Kaffee, and Isabelle Augenstein (2023). Probing Pre-Trained Language Models for Cross-Cultural Differences in Values. Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), 114–130. Association for Computational Linguistics. ↩
-
Yan Tao, Olga Viberg, Ryan S Baker, and René F Kizilcec (2024). Cultural bias and cultural alignment of large language models. PNAS Nexus, 3(9), pgae346. Preprint: arXiv:2311.14096. ↩ ↩2 ↩3
-
Maksim E. Eren, Eric Michalak, Brian Cook, and Johnny Seales Jr. (2026). Prompt Programming for Cultural Bias and Alignment of Large Language Models. Proceedings of the 2026 ACM Symposium on Document Engineering, 1–10. ACM. arXiv:2603.16827. ↩
-
EVS (2022). EVS Trend File 1981–2017. GESIS Data Archive, Cologne. ZA7503 Data file Version 3.0.0. ↩ ↩2
-
Christian Haerpfer, Ronald Inglehart, Alejandro Moreno, Christian Welzel, Kseniya Kizilova, Jaime Diez-Medrano, Marta Lagos, Pippa Norris, Eduard Ponarin, and Bi Puranen (editors) (2022). World Values Survey Trend File (1981–2022) Cross-National Data-Set. Madrid, Spain & Vienna, Austria: JD Systems Institute & WVSA Secretariat. Data File Version 4.0.0 (2024-06-30); author-reported retrieval 11 August 2024. ↩ ↩2 ↩3
-
A. Colin Cameron, Jonah B. Gelbach, and Douglas L. Miller (2008). Bootstrap-based improvements for inference with clustered errors. The Review of Economics and Statistics, 90(3), 414–427. ↩ ↩2
-
James Luther and Donald Brown (2025). DeepSeek's WEIRD Behavior: The cultural alignment of Large Language Models and the effects of prompt language and cultural prompting. arXiv preprint. arXiv:2512.09772. ↩
-
Elise Karinshak, Amanda Hu, Kewen Kong, Vishwanatha Rao, Jingren Wang, Jindong Wang, and Yi Zeng (2024). LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Output. arXiv preprint. arXiv:2411.06032. ↩
-
Bram Bulté and Ayla Rigouts Terryn (2026). LLMs and Cultural Values: The Impact of Prompt Language and Explicit Cultural Framing. Computational Linguistics, 52(2), 407–494. arXiv:2511.03980. ↩
-
Kun Sun and Rong Wang (2025). The fragility of ``cultural tendencies'' in LLMs. arXiv preprint. arXiv:2510.05869. ↩
-
Jackson G. Lu, Lesley Luyang Song, and Lu Doris Zhang (2025). Cultural tendencies in generative AI. Nature Human Behaviour, 9(11), 2360–2369. ↩
-
Sharif Kazemi, Gloria Gerhardt, Jonty Katz, Caroline Ida Kuria, Estelle Pan, and Umang Prabhakar (2024). Cultural Fidelity in Large-Language Models: An Evaluation of Online Language Resources as a Driver of Model Performance in Value Representation. arXiv preprint. arXiv:2410.10489. ↩ ↩2
-
Gabriele Maraia, Fabio Massimo Zanzotto, and Leonardo Ranaldi (2026). Sounding vs. Being an Expert: Disentangling Authority, Register and Cultural Impact in Sycophantic LLMs. Findings of the Association for Computational Linguistics: ACL 2026, 32492–32508. Association for Computational Linguistics. ↩
-
Jens Rupprecht, Georg Ahnert, and Markus Strohmaier (2026). Prompt Perturbations Reveal Human-Like Biases in Large Language Model Survey Responses. Proceedings of the Seventh Workshop on Natural Language Processing and Computational Social Science (NLP+CSS 2026), 1–21. Association for Computational Linguistics. arXiv:2507.07188. ↩
-
Rom Himelstein, Amit LeVi, Brit Youngmann, Yaniv Nemcovsky, and Avi Mendelson (2026). Silenced Biases: The Dark Side LLMs Learned to Refuse. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI 2026), AI Alignment track. Oral. Preprint 2025, arXiv:2511.03369. ↩
-
Wei Xie, Shuoyoucheng Ma, Zhenhua Wang, Xiaobing Sun, Kai Chen, Enze Wang, Wei Liu, and Hanying Tong (2025). AIPsychoBench: Understanding the Psychometric Differences between LLMs and Humans. Proceedings of the Annual Meeting of the Cognitive Science Society (CogSci 2025), 87–94. ↩
-
Charles F. Manski (2003). Partial Identification of Probability Distributions. Springer. ↩ ↩2
-
World Values Survey Association. Building the Tradrat and Survself factors. WVS Database documentation (SPSS syntax:
SurvSAgg = 1.81 * SurvSelf + .038,TradAgg = 1.61 * TradRat5 - .1). ↩ -
Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, Vlad Mikulik, Samuel R. Bowman, Jan Leike, Jared Kaplan, and Ethan Perez (2025). Reasoning Models Don't Always Say What They Think. arXiv preprint. arXiv:2505.05410. ↩
-
Dai Li, Linzhuo Li, and Huilian Sophie Qiu (2025). ChatGPT is not A Man but Das Man: Representativeness and Structural Consistency of Silicon Samples Generated by Large Language Models. arXiv preprint. arXiv:2507.02919. ↩
-
Erika Elizabeth Taday Morocho, Lorenzo Cima, Tiziano Fagni, Marco Avvenuti, and Stefano Cresci (2026). Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondents. Companion Proceedings of the ACM Web Conference 2026, 320–329. arXiv:2602.18462. ↩
-
An Duy Nguyen and Muhammad Aurangzeb Ahmad (2026). Measurement Validity in LLM Cultural Alignment. arXiv preprint. arXiv:2608.29266. ↩
-
Petter Törnberg and Michelle Schimmel (2026). Political Bias Audits of LLMs Capture Sycophancy to the Inferred Auditor. arXiv preprint. arXiv:2604.27633. ↩
-
Jared Moore, Tanvi Deshpande, and Diyi Yang (2024). Are Large Language Models Consistent over Value-laden Questions?. Findings of the Association for Computational Linguistics: EMNLP 2024, 15185–15221. arXiv:2407.02996. ↩
-
Grgur Kovač, Rémy Portelas, Masataka Sawayama, Peter Ford Dominey, and Pierre-Yves Oudeyer (2024). Stick to your Role! Stability of Personal Values Expressed in Large Language Models. PLOS ONE, 19(8), e0309114. arXiv:2402.14846. ↩
-
Haonan Huang (2026). The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgment. arXiv preprint. arXiv:2607.05552. ↩
-
Trung Duc Anh Dang, Tung Kieu, and Sarah Masud (2026). Scenario-based Probing and Steering Cultural Values in Large Language Models - Extended Version. arXiv preprint. arXiv:2606.11399. ↩ ↩2
-
Trung Duc Anh Dang and Sarah Masud (2026). Cultural Value Alignment Via Latent Activation Steering in Large Language Models. arXiv preprint. arXiv:2605.26365. Presented at the ACL 2026 Student Research Workshop (non-archival track). ↩ ↩2 ↩3
-
Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto (2023). Whose Opinions Do Language Models Reflect?. Proceedings of the International Conference on Machine Learning (ICML 2023), 202, 29971–30004. arXiv:2303.17548. ↩ ↩2
-
Michael J. Ryan, William Held, and Diyi Yang (2024). Unintended Impacts of LLM Alignment on Global Representation. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), 16121–16140. arXiv:2402.15018. ↩
-
David Rozado (2024). The political preferences of LLMs. PLOS ONE, 19(7), e0306621. ↩
-
Rochelle Choenni, Anne Lauscher, and Ekaterina Shutova (2024). The Echoes of Multilinguality: Tracing Cultural Value Shifts during Language Model Fine-tuning. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), 15042–15058. arXiv:2405.12744. ↩
-
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems (NeurIPS 2022). arXiv:2203.02155. ↩
-
Saffron Huang, Divya Siddarth, Liane Lovitt, Thomas I. Liao, Esin Durmus, Alex Tamkin, and Deep Ganguli (2024). Collective Constitutional AI: Aligning a Language Model with Public Input. Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT 2024), 1395–1417. ↩
-
Thom Lake, Eunsol Choi, and Greg Durrett (2025). From Distributional to Overton Pluralism: Investigating Large Language Model Alignment. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL 2025), 6794–6814. arXiv:2406.17692. ↩
-
Maarten Buyl, Alexander Rogiers, Sander Noels, Guillaume Bied, Iris Dominguez-Catena, Edith Heiter, Iman Johary, Alexandru-Cristian Mara, Raphaël Romero, Jefrey Lijffijt, and Tijl De Bie (2026). Large language models reflect the ideology of their creators. npj Artificial Intelligence, 2(1), 7. arXiv:2410.18417. ↩
-
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Hanwei Xu, Honghui Ding, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jingchang Chen, Jingyang Yuan, Jinhao Tu, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaichao You, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingxu Zhou, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wangding Zeng, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang (2025). DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature, 645(8081), 633–638. Distillation provenance is addressed in the supplementary materials. ↩
-
Mingjie Sun, Yida Yin, Zhiqiu Xu, J Zico Kolter, and Zhuang Liu (2025). Idiosyncrasies in Large Language Models. Proceedings of the International Conference on Machine Learning (ICML 2025), 267, 57854–57885. arXiv:2502.12150. ↩
-
Luísa Shimabucoro, Sebastian Ruder, Julia Kreutzer, Marzieh Fadaee, and Sara Hooker (2024). LLM See, LLM Do: Leveraging Active Inheritance to Target Non-Differentiable Objectives. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP 2024), 9243–9267. arXiv:2407.01490. ↩
-
Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song (2023). The False Promise of Imitating Proprietary LLMs. arXiv preprint. arXiv:2305.15717. ↩
-
Liwei Jiang, Yuanjun Chai, Margaret Li, Mickel Liu, Raymond Fok, Nouha Dziri, Yulia Tsvetkov, Maarten Sap, and Yejin Choi (2025). Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond). Advances in Neural Information Processing Systems (NeurIPS 2025), Datasets and Benchmarks Track. NeurIPS 2025 proceedings version. ↩
-
Candida M. Greco, Lucio La Cava, and Andrea Tagarelli (2026). Culturally Grounded Personas in Large Language Models: Characterization and Alignment with Socio-Psychological Value Frameworks. arXiv preprint. arXiv:2601.22396. ↩
-
Xunlian Dai, Li Zhou, Benyou Wang, and Haizhou Li (2025). From Word to World: Evaluate and Mitigate Culture Bias in LLMs via Word Association Test. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP 2025), 24510–24526. Oral. arXiv:2505.18562. ↩
-
Qian Tan, Lei Jiang, Yuting Zeng, Shuoyang Ding, and Xiaohua Xu (2026). Mitigating Cultural Bias in LLMs via Multi-Agent Cultural Debate. Findings of the Association for Computational Linguistics: ACL 2026, 8600–8612. Association for Computational Linguistics. arXiv:2601.12091. ↩
-
Terry Kit-Fong Au (1983). Chinese and English counterfactuals: The Sapir–Whorf hypothesis revisited. Cognition, 15(1–3), 155–187. ↩
-
Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, Alex Tamkin, Esin Durmus, Tristan Hume, Francesco Mosconi, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan (2024). Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet. Transformer Circuits Thread. ↩
-
Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, Nicholas L. Turner, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, Andrew Persic, Zhenyi Qi, T. Ben Thompson, Sam Zimmerman, Kelley Rivoire, Thomas Conerly, Chris Olah, and Joshua Batson (2025). On the Biology of a Large Language Model. Transformer Circuits Thread. ↩
-
Emmanuel Ameisen, Jack Lindsey, Adam Pearce, Wes Gurnee, Nicholas L. Turner, Brian Chen, Craig Citro, David Abrahams, Shan Carter, Basil Hosmer, Jonathan Marcus, Michael Sklar, Adly Templeton, Trenton Bricken, Callum McDougall, Hoagy Cunningham, Thomas Henighan, Adam Jermyn, Andy Jones, Andrew Persic, Zhenyi Qi, T. Ben Thompson, Sam Zimmerman, Kelley Rivoire, Thomas Conerly, Chris Olah, and Joshua Batson (2025). Circuit Tracing: Revealing Computational Graphs in Language Models. Transformer Circuits Thread. ↩
-
Taisei Yamamoto, Ryoma Kumon, Danushka Bollegala, and Hitomi Yanaka (2026). Neuron-Level Analysis of Cultural Understanding in Large Language Models. Proceedings of the International Conference on Learning Representations (ICLR 2026). arXiv:2510.08284. ↩
-
Jongwook Han, Jongwon Lim, Injin Kong, and Yohan Jo (2026). Dual Mechanisms of Value Expression: Intrinsic vs. Prompted Values in Large Language Models. Proceedings of the International Conference on Machine Learning (ICML 2026). arXiv:2509.24319. ↩
-
Simran Khanuja, Hongbin Liu, Shujian Zhang, John Lambert, Mingqing Chen, Rajiv Mathews, and Lun Wang (2026). Steering LLMs for Culturally Localized Generation. arXiv preprint. arXiv:2603.23301. ↩
-
Veniamin Veselovsky, Berke Argın, Benedikt Stroebl, Chris Wendler, Robert West, James Evans, Thomas L. Griffiths, and Arvind Narayanan (2026). Localized Cultural Knowledge is Conserved and Controllable in Large Language Models. Findings of the Association for Computational Linguistics: ACL 2026, 43152–43178. arXiv:2504.10191. ↩
-
Clément Dumas, Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West (2025). Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformers. arXiv:2411.08745v4, revised 25 June 2025. ↩
-
Cheng-Ting Chou, George Liu, Jessica Sun, Cole Blondin, Kevin Zhu, Vasu Sharma, and Sean O'Brien (2025). Causal Language Control in Multilingual Transformers via Sparse Feature Steering. arXiv:2507.13410v2, revised 15 October 2025. ↩
-
Zhengxuan Wu, Aryaman Arora, Atticus Geiger, Zheng Wang, Jing Huang, Dan Jurafsky, Christopher D Manning, and Christopher Potts (2025). AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders. Proceedings of the International Conference on Machine Learning (ICML 2025), 267, 67035–67080. arXiv:2501.17148. ↩
-
David Chanin, James Wilken-Smith, Tomáš Dulka, Hardik Bhatnagar, Satvik Golechha, and Joseph Bloom (2025). A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders. Advances in Neural Information Processing Systems (NeurIPS 2025). Oral. arXiv:2409.14507. ↩
-
Joshua Engels, Logan Smith, and Max Tegmark (2025). Decomposing The Dark Matter of Sparse Autoencoders. Transactions on Machine Learning Research. ↩
-
Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeff Wu, Lucius Bushnaq, Nicholas Goldowsky-Dill, Stefan Heimersheim, Alejandro Ortega, Joseph Bloom, Stella Biderman, Adria Garriga-Alonso, Arthur Conmy, Neel Nanda, Jessica Rumbelow, Martin Wattenberg, Nandi Schoots, Joseph Miller, William Saunders, Eric J. Michaud, Stephen Casper, Max Tegmark, David Bau, Eric Todd, Atticus Geiger, Mor Geva, Jesse Hoogland, Daniel Murfet, and Tom McGrath (2025). Open Problems in Mechanistic Interpretability. Transactions on Machine Learning Research. ↩
-
Michael E. Tipping and Christopher M. Bishop (1999). Probabilistic principal component analysis. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 61(3), 611–622. ↩
-
Allen Tran (2015). pca-magic. GitHub repository. Apache-2.0 licensed. ↩
-
Karl Rohe and Muzhe Zeng (2023). Vintage factor analysis with Varimax performs statistical inference. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 85(4), 1037–1060. ↩ ↩2
-
Roderick Little and Donald Rubin (2019). Statistical Analysis with Missing Data. 3rd edition. Wiley. ↩