Executive Summary

Key Numbers

Source: the result figures and tables in arXiv:2609.00192. In the paper, the yield rate is the share of times a model answers that it would stop.

Autonomous-driving research grew a new branch a few years ago. Rather than write out rules one at a time, it brings in a general-purpose model that already knows how the world works and hands it the driving call. The paper cites four earlier works in that branch. They ask and answer questions in language about what the camera saw, treat a driving scene as graph question answering, attach vision-language representations to 3D scene understanding, and wire a large vision-language model into end-to-end driving.

Three reasons make the approach attractive, as the paper lays them out. A model can produce judgment close to a person's. It can handle situations and edge cases nobody programmed explicitly, using knowledge it already holds. And the burden of collecting real driving data goes down.

The trouble lies in where that common sense came from. Reports that large language models and vision-language models hold bias and discriminatory stereotypes have piled up already, and other research shows those stereotypes getting enacted as behavior once the models drive robot control. This paper took its age and disability labels straight from that robotics work.

The evidence about human drivers is older. In the 2015 crosswalk study the paper cites, drivers in the US yielded less often to Black pedestrians than to white pedestrians, and Black pedestrians waited 32% longer as a result. The same paper's related-work section puts that same source at roughly 30% longer waits. Observations that a pedestrian's condition, gender, age, and eye contact with the driver change yielding behavior appear alongside it.

The two lines of evidence meet at one question. Did a model that learned human common sense learn the human driver's bias too? Gathering this evidence before deployment keeps trust from being lost later, the authors write. That is the paper's aim.

To measure bias you have to hold every other condition fixed and vary demographics alone. Photographs do not allow it. No public dataset holds image pairs that keep the same location, the same camera angle, the same surroundings and the same number of pedestrians while the pedestrian's ethnicity or gender changes. That absence is why the authors split their method in two.

The starting point is shared. From three public datasets, nuImages, JAAD and PIE, they collected every image with exactly one pedestrian marked by a bounding box, since several pedestrians in one frame let the characteristics interfere with each other. The 23,812 images gathered this way became the common base for both tests.

2.1Scenarios With One Line Swapped In

The first test serves language models, and the paper calls it the AEBE (All Else Being Equal) test. Out of the 23,812 images the authors hand-picked 3,346 showing a pedestrian close to the car with visible intent to cross. Intent was judged by whether the pedestrian actually started crossing in the next frame.

Qwen-3-VL turned the selected photos into text descriptions. The prompt asked the model to imagine explaining the traffic scene to a driver, to concentrate on the pedestrian's movement and crossing attempt, and to keep it within five sentences. The next step is the labor-intensive one. Wherever the model had written in a gender nobody asked for, a person went in and removed it, 55 times for female and 203 for male. Scenes whose wording already carried the sense that the driver ought to stop came out too. That left 3,157 scenarios with no demographic information in them.

Demographics then go back in. Gender, ethnicity, religion, disability, age, skin tone, socio-economic status, and gender paired with each of those. They keep an 'original' condition with no demographics written in as well. Because only one line changes in the same scene, everything else stays exactly equal. The label lists were not invented for the occasion: they come from the US Census and the CIA World Factbook, from the earlier robotics work, and from FairFace, a face-attribute dataset. Labels commonly assumed to be offensive, 'Old' and 'Normal' among them, were left in on purpose. The paper's stated intent was to examine what a commonly used but potentially offensive word choice does to the result.

The models get one question. Should this autonomous vehicle stop for this pedestrian? Yes or no. To pin the answer down, the authors pulled the probability of each answer from the model's log-odds output and took the more probable one as the decision. That removes the wobble of a different answer each time the same question runs. Conditions built this way run from 12,628 for gender to 176,792 for gender paired with disability, and the ten rows of Table 1 add up to 584,045.

2.2The Model's Own Consistency

The second test serves vision models and goes by the name self-consistency. Since photographs cannot be equalized, the authors handed the judgment of 'same conditions' to the model itself. The model is first asked whether this pedestrian intends to cross, and whether the car stopping is both required and sufficient for that pedestrian to cross safely. Only scenes answered yes on both counts stay in.

For the remaining scenes the model faces two separate questions. Should the car stop, and what demographic characteristics does this pedestrian appear to have? The premise runs like this: among scenes the model itself judged to require a stop, the stop decision ought to hold steady whatever the demographics. A decision that does not hold steady is itself evidence of bias. Every element of the test gets computed by the model under evaluation, which is where the name came from.

The authors put on record that the two tests do not carry equal weight. The self-consistency test is not as robust as the equal-conditions comparison and may be influenced by scene context and the model's reasoning capabilities. Running the equal-conditions comparison on vision models as well would take a dataset of images differing only in demographics, and the authors leave that to future work, reachable through human actors or through advances in synthetic data generation.

Four models sat on the language side: Qwen-3, Llama-3.1, Mistral and GPT-4o. The first three are open models at 8B and 7B, picked with an eye to what can run inside a car. GPT-4o came in through an API and is far larger. All four answered with no examples given beforehand.

Taken as a whole, the models mostly do stop. Three of the four hold a median near 100% across every demographic condition. Only Qwen-3 runs low, with a median around 70%, and its variance is the largest of the four. At the bottom end of that spread lies one label. Paralyzed.

Qwen-3's yield rate by disability runs highest at 99.2% when disability status reads 'unclear' and lowest at 8.2% for a paralyzed pedestrian. Twelve labels stack up in between. The three labels reading able bodied, normal and nondisabled gather near the top at 97.2%, 97.0% and 95.0%, and under them come the original with no demographics at 91.6%, amputee at 91.3%, Down syndrome at 83.2%, deaf at 80.3%, blind at 79.2%, wheelchair user at 78.1%, autistic at 74.9%, ADHD at 71.7% and nonspeaking at 69.8%. Almost every pairwise difference came out statistically significant on a chi-square test.

This is no quirk of Qwen-3 alone. Llama-3.1 held between 99.6% and 100% across every disability label and then sank to 95% for paralyzed, the one position that differed significantly from all the others. Its other variables varied so little that the authors omitted the graph entirely. Mistral likewise dropped its yield rate for several groups tied to disability and age. The paper records that the group 'paralyzed' received lower yield rates across all models.

Other axes take a different shape from model to model. Qwen-3 stopped least often for female pedestrians at 61.1%, against 67.4% when gender was unclear, 67.9% for male, and 91.6% when gender went unwritten. Mistral ran the other way, with female at 98.2% above male at 97.1%. The difference is small, but the sample is large enough that it counts as statistically significant. For GPT-4o most pairwise differences did not pass the chi-square test. Some small gaps did pass it, and the example the paper offers is religion. Jewish comes highest at 99.5%, with Muslim at 99.0%, Christian at 98.9% and Sikh at 98.9% significantly below it. The whole spread stays under one percentage point.

The values quoted so far do not spread evenly over the seven axes, however. Four figures on the language-model side carry numbers, and they cover three axes: gender, disability and religion. For age, ethnicity, skin tone and socio-economic status there is nowhere in the paper's body to read a language-model value. The choices offered for socio-economic status were wealthy, not well off financially, and unclear. The authors write that Qwen-3 is significantly biased across all seven axes, and most of the figures holding that sentence up live in an outside repository rather than on the page.

Combined conditions stacked the gaps on top of one another. Qwen-3 showed markedly lower yield rates for particular combinations naming gender together with disability, gender with ethnicity, and gender with age. The authors read this as a compounding effect of discrimination over multiple dimensions, and as the reason intersectional analysis matters.

Yield rates for the four vision models run far below the language models. The medians are 30% for Qwen-3-VL, 15% for SPHINX, 8% for Qwen-2.5-VL and 1% for LLaVA-NeXT. These numbers should not be read next to the language models' 70% to 100%. The self-consistency test keeps only the scenes a model itself called a required stop and then asks again, so the absolute figures carry a different meaning. The part worth a second look is elsewhere. In scenes where it had just said the pedestrian intended to cross and a stop was needed, LLaVA-NeXT answered that it would stop once in a hundred times.

Skin tone shows up most sharply in Qwen-2.5-VL. Fair white 6.7%, brown 6.2%, medium 4.5%, the darkest tone 0.0%, with the values lined up in one direction. Statistically significant differences came between fair white and medium, and between fair white and the darkest. This slope matches the direction of the human-bias studies conducted in the US, the authors write. A shape observed in human drivers came back out of a model.

Ethnicity moved more in the other vision models. LLaVA-NeXT said it would stop in 50.0% of the scenes where it estimated the pedestrian as Indian, and in under 4% for every other ethnicity it estimated. One label jumped more than tenfold. In SPHINX, Indian also averaged highest at 21.4% while drawing no stars at all, and Black at 20.0% and Southeast asian at 18.9% ranked significantly above White at 10.2%. Hispanic and East asian did not reach 5%. Qwen-3-VL split on gender, with female at 34.6% above male at 28.2%, and that difference passed the test too.

Which label jumps to the top depends on the model. The paper notes this separately. High-yield outliers in three of the vision models all involved ethnicity, Indian for LLaVA-NeXT and SPHINX, Hispanic for Qwen-3-VL. In SPHINX that same Hispanic label was near the floor at 4.9%. Who loses out shifts as the model shifts. An ordering obtained from one model cannot be copied onto another model.

The list of choices deserves one more look. The ethnicity options hold both 'Asian' and 'East Asian'. Which of the two a person in a photo gets called is the model's pick, and SPHINX's yield rate split between 13.9% for Asian and 3.2% for East asian. This comparison is not in the paper. In the self-consistency test the demographics are values the model produced rather than values it received, and where the option list got cut apart divides those values. Designing the axes is already part of the result.

4.1The Model That Refused Got Dropped

One more name appears in the vision-model table: GPT-4o Vision. That model refused to answer demographic questions about the pedestrians in the images, gender and ethnicity among them, and was therefore excluded from the vision tests altogether. In the paper it gets a single footnote.

From here on this is the article's reading. Refusing to estimate demographics is, taken on its own, a well-designed safeguard. Better to have a model that declines to rule on the race of a person in a photo. But the self-consistency test measures bias with the model's own demographic estimates as its axis, so the moment a model refuses to estimate, its fairness leaves the set of things that can be measured. Whether this model stops differently depending on how a pedestrian looks went unconfirmed. The tighter the guardrail, the narrower the path to checking from outside how fair that model really is. To whoever decides on adoption, an unmeasured model and an unbiased model land in the same box on the report card.

One distinction has to stay in place. The model dropped from the test was GPT-4o Vision, which handles images, and the GPT-4o tested with text scenarios remains on the language-model side. The paper also records that in every model including GPT-4o, the yielding decision was significantly associated with the pedestrian's personal characteristics.

The paper's conclusion carries one sentence aimed at mitigation. On average, models tended to predict yielding decisions more often when no information about pedestrian demographics was provided, or when identity predictions came back 'unclear', which demonstrates the positive potential of filtering methods that remove demographic identifiers or proxies from the input, or of model activation steering.

A sentence like that invites quotation. So this article went and counted how far the rule holds in the figures the paper actually published. Four language-model figures appear in the paper, and in all four the no-demographics condition carries the name 'original'.

• In Qwen-3's gender figure, original at 91.6% is the highest of the four conditions. The rule holds.
• In the same model's disability figure, 'unclear' at 99.2% is highest, while original at 91.6% ranks fifth among fourteen conditions. All three labels meaning non-disabled rank above original.
• In Mistral's gender figure, original at 93.3% is the lowest of the four conditions. The paper's own prose notes that for this model the absence of gender information lowers the yield rate.
• In GPT-4o's religion figure, original at 98.1% is the lowest of six conditions.

Original comes out highest in one figure of the four, and lowest in two. 'Unclear' fares much the same, highest in only one of the four, the disability figure. That much is read straight off those four figures.

The authors drew a line of their own as well. The focus of the work was assessment and not mitigation, they write, and the root causes of these biases and the effects of mitigation strategies remain uninvestigated. Few-shot evaluation, Chain-of-Thought, filtering and activation steering are named on that list. This is a paper that built the ruler for testing mitigations, not a paper that tested them.

A further layer belongs here, one the paper does not carry. The text scenarios used in the language-model test were written by Qwen-3-VL, and that same model is a subject in the vision-model test. The model that wrote the exam and the models that sat it overlap inside one family. The authors reviewed that text by hand and stripped the demographic mentions, which takes much of the risk away. Still, the words each scene got turned into were chosen by a model.

The model leaderboard will not be the part of this paper that lasts. Rankings change with the next release. Two methods will last longer. Both got around the constraint Section 2 set out: photographs cannot hold the other conditions equal. One moved the scene into text to control the conditions, and the other took the model's own judgment as the condition. Neither one came from collecting more data.

In work on data quality that point carries. Talk of fixing bias usually drifts toward collecting more data or attaching more labels. The figure 8.2% held in this study not because the data was plentiful but because somebody built pairs of scenarios differing in one line and identical in everything else. A quality check turns on what you built so that it could be compared, and not on how much you collected.

The call to make bias tests part of the evaluation came from the authors first. The paper calls for the use of AEBE, SC and similar bias tests in models used for AV decisions, and adds that developers applying general-purpose models to AVs should seriously consider bias as another factor in model evaluation and safeguarding. The last sentence of the abstract reaches further. It raises the question of whether the 'common sense' model paradigm needs revising, or whether the paradigm stays and downstream bias gets addressed instead. The paper does not answer that for us.

Just as this paper set demographics up as an axis, the models we use need axes of the same kind. When people at work say an in-house AI tool is doing well or doing badly, four questions bring out roughly where things stand. Carried outside autonomous driving, these four are questions the paper does not ask.

• Do this tool's evaluation items include even one axis that divides people? With nothing but average accuracy written down, nobody knows who loses out inside that average.
• To measure that axis, have input pairs been built with every other condition held equal? Without them, there is no telling whether a difference in results came from bias or from a difference in conditions.
• Is an item the model refused to answer being counted as a pass? Avoiding the measurement is not the same as having no problem.
• Is erasing sensitive information from the input standing in for this item? As Section 5 showed, the condition with the information erased was not always the fairer one.

The fourth is the cheapest lesson to carry away from this study. Taking gender or race out of the input costs little effort and explains itself easily, so it tends to land first among bias measures. And in two of the four figures from the previous section, yield rates were lowest when the information had been erased. A box filled in because the data was erased is a box where nothing was measured.

Autonomous driving is only the stage where this story shows most clearly. The same structure runs through hiring reviews, loan reviews, automated customer service. The difference at a crosswalk is that the outcome translates straight into waiting time and accident risk. A pedestrian yielded to less often waits longer, and a pedestrian who waits longer crosses into a riskier gap.

Thank you for reading this far. The numbers and sentences this article cites can be checked by anyone in arXiv:2609.00192, and the figures that did not fit in the paper are in the authors' public repository. The comparison in Section 5 is this article's own count off the four figures printed in the paper. If the look of a person crossing the street changes the judgment, we wonder whether the report card for the model you use carries that item. If it does not, we would be glad to hear what is standing in the way.

Primary Source

• 1.Yoldas, I., Brandão, M., Zhang, J., Rodrigues, O. (2026). "LLM-Driven Autonomous Vehicles Inherit Human Driver Biases in Pedestrian Yielding: Results and Implications From A New Benchmark." arXiv:2609.00192.
• 2.Yoldas, I. "LLM-Driven-AV-Pedestrian-Bias" (public repository of every model's and demographic variable's figures). GitHub.

Prior Work

• 3.Goddard, T., Kahn, K. B., Adkins, A. (2015). "Racial bias in driver yielding behavior at crosswalks." Transportation Research Part F: Traffic Psychology and Behaviour, 33, 1-6.
• 4.Hundt, A., Agnew, W., Zeng, V., Kacianka, S., Gombolay, M. (2022). "Robots enact malignant stereotypes." Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT '22), 743-756.
• 5.Hundt, A., Azeem, R., Mansouri, M., Brandão, M. (2024). "LLM-driven robots risk enacting discrimination, violence, and unlawful actions." arXiv:2406.08824.