We’ve all been told not to judge a book by its cover, and you’d think AI would follow that rule, too. Ask an AI chatbot to guess someone’s sexual orientation just by looking at their face, and it’ll likely tell you that there’s no reliable way to know. But what happens when you ask that same model to change someone’s appearance to match a particular sexual orientation? Well, that’s where things get uncomfortable.
A new study, accepted for presentation at the 2026 Conference on Empirical Methods in Natural Language Processing, found that AI models from OpenAI and Google can be inconsistent when handling sensitive personal characteristics. They might refuse to make assumptions about someone based on their appearance, but when asked to generate those same assumptions visually, the results tell a very different story.
AI knows the rules, but doesn’t always follow them
Researchers tested OpenAI’s GPT Image 1 Mini and Google’s Gemini 2.5 Flash Image, better known as Nano Banana, using 1,002 AI-generated faces. Initially, both models behaved as expected. When asked whether someone looked gay or straight, they refused to make that judgment. Even when researchers presented two faces and asked which person was more likely to be gay, Gemini declined 92% of the time, while GPT refused 91% of the time.
But things changed dramatically when the researchers switched from asking questions to requesting image edits. When instructed to make someone appear gay or straight, GPT followed through more than 70% of the time. Gemini’s compliance rate was even higher, exceeding 99%. And it didn’t stop there. The researchers also asked the models to alter people’s appearances based on racial and ethnic categories. After analyzing 14,131 modified images, another AI system identified the intended sexual-orientation categories with 83% to 88% accuracy, suggesting the models repeatedly introduced recognizable visual patterns.
The bigger problem isn’t just the pictures
The researchers then asked the models to describe the personalities, interests, and occupations of people in the edited images. Unsurprisingly, more stereotypes emerged. Images modified to appear gay were more frequently associated with fashion and theater, while those modified to appear straight were more often linked to sports. In another experiment, both models complied with requests to make people look as though they had criminal records, or didn’t, more than 97% of the time.
That’s concerning because someone’s face cannot reliably reveal their sexuality, personality, or criminal history. Yet these AI systems were effectively creating visual shortcuts for characteristics that shouldn’t be determined by appearance. There are limitations worth considering. The researchers used synthetic faces rather than photographs of real people, and it’s unclear whether the results would translate directly to real-world images. Still, the findings highlight an important AI safety problem. Preventing a chatbot from making an inappropriate statement is one thing, but ensuring it doesn’t communicate the same harmful assumptions through generated images is another challenge entirely.