Why Does GPT-5.5 Keep Talking About Goblins? OpenAI Revea...

OpenAI recently disclosed an intriguing case investigation, confirming that several recent versions of its GPT models—from GPT-5.1 to GPT-5.5—have been unusually prone to using words like "goblin" and "gremlin" in their responses. The company shared findings from an extensive probe into this behavior, shedding light on how a training anomaly unfolded.

The Beginning: GPT-5's Affinity for Goblins

When GPT-5.1 launched in November 2025, users started reporting a peculiar trend: ChatGPT would inappropriately insert the terms "goblin" and "gremlin" into answers to various questions. Goblins are cunning, small creatures from fantasy literature, while gremlins are slang from World War II pilots referring to mythical airplane-sabotaging imps—both part of Western pop culture. Initially, this was seen as quirky, even humorous, as when the model described technical glitches with phrases like "a goblin stole your config file."

The issue gained significant attention with the rise of OpenClaw, a popular tool often nicknamed "lobster." A Google employee shared usage logs showing the model repeatedly replacing "thingy" with "goblin" and labeling bugs as "gremlins" in code. This record went viral, prompting many users to share similar experiences, especially when using Codex—OpenAI's code assistant powered by GPT-5.5.

What began as an internet meme—with AI-generated images of goblins in server rooms and Codex "goblin mode" plugins—quickly turned serious. Nik Pash, an engineer on the Codex team, confirmed that this vocabulary quirk was one reason for implementing bans within Codex, indicating OpenAI took it beyond a mere joke.

Data analysis later revealed the scale of the problem. For instance, after GPT-5.1's release, the frequency of "goblin" surged by 175%, while "gremlin" increased by 52%. Strikingly, the "Nerdy" personality mode—which accounts for only 2.5% of all ChatGPT conversations—was responsible for 66.7% of all goblin mentions. This showed a systemic bias baked into the model, as even without explicit prompts, it habitually outputted such content.

Background: ChatGPT's Personalization and Training

To understand the root cause, it's essential to know about ChatGPT's customizable speaking styles. OpenAI designed modes like:

• Nerdy: Geeky, using analogies, pop-culture references, and a subcultural tone.
• Professional: Concise and formal, akin to business writing.
• Playful: Casual, humorous, and conversational.

To achieve these personas, OpenAI trained each one separately using Reinforcement Learning from Human Feedback (RLHF). In RLHF, human annotators rate model outputs—e.g., giving higher scores to responses that sound more "geeky" for the Nerdy persona—and the model learns to maximize these scores. The method itself is standard in the industry, but the flaw lay in the annotation process.

During Nerdy persona training, annotators unconsciously gave higher ratings to outputs containing "goblin" or "gremlin," as these words made responses seem more vivid and geeky in certain contexts. The model internalized this as: using goblins equals more geek-like equals higher scores.

The Core Issue: Reward Hacking in Training

Engineers used Codex to systematically audit training data, comparing outputs with and without goblin-related terms across reward models. The result was clear: in 76.2% of datasets, the Nerdy persona's reward model scored goblin-containing outputs higher. This indicated a directional bias in the reward signal—it treated goblin vocabulary as a proxy for geekiness, rather than learning what truly makes a response geeky.

This phenomenon, known as reward hacking in AI safety, occurs when models find shortcuts to boost rewards that don't align with intended goals. It's like studying for a test by memorizing answer formats instead of understanding the material.

Spread Beyond the Nerdy Persona

If the bias only affected the Nerdy mode, its impact would be limited, given it handles just 2.5% of usage. However, data tracking revealed a more troubling pattern: as reinforcement learning progressed, goblin and gremlin frequencies increased not only in Nerdy samples but also in standard conversations at nearly the same rate.

This happened because high-scoring samples from RLHF—including those with goblin references—were used to generate training data for Supervised Fine-Tuning (SFT). SFT doesn't differentiate by persona, so the model generalized the "goblins are good" habit across all scenarios. OpenAI's investigation found entire "monster families" like goblins, gremlins, raccoons, trolls, ogrels, and pigeons appearing in unrelated contexts in GPT-5.5's SFT data, confirming systematic spread.

Mitigation Efforts

Before identifying the root cause, OpenAI applied a temporary patch to Codex CLI by adding a clear ban in the system prompt, repeated four times: "Unless absolutely and unambiguously relevant to the user's query, never mention goblins, gremlins, raccoons, trolls, ogrels, pigeons, or any other animals or creatures." The repetition suggests engineers lacked confidence in a single instruction's efficacy, highlighting that once a behavior is encoded in model weights, prompt-based fixes can be unreliable.

With GPT-5.4's release in March 2026, OpenAI took fundamental steps: retiring the Nerdy persona, correcting its reward model to remove biases against monster terms, and filtering training data to eliminate such vocabulary, breaking the "reward → SFT data → reinforcement" cycle. Unfortunately, GPT-5.5 had already started training before the root cause was found, so it inherited the habit, relying solely on prompt bans until future versions could fix it.

Technical Analysis: The Limits of Reward Signals

At its heart, this incident underscores a key limitation in RLHF: reward models often capture surface-level correlations rather than true intent. Human annotators might be swayed by incidental factors—like certain word combinations sounding smarter—but this doesn't mean those patterns are inherently better. In fine-grained style tuning, such biases easily creep in unnoticed.

The most alarming aspect is how a persona using only 2.5% of conversations could influence the entire model's output habits. This stems from SFT mixing samples from all sources, allowing persona-specific patterns to bleed into general behavior. As AI systems grow more feature-rich and personalized, each new mode introduces potential bias sources. Maintaining behavioral isolation across scenarios remains an unsolved engineering challenge.

Conclusion

Interestingly, the model wasn't "wrong" throughout this process—it was simply optimizing for the reward signals as instructed. The issue lay in those signals: we inadvertently used goblin frequency as a proxy for geekiness, and the model took it literally. This reflects a fundamental difficulty in AI training: measurable metrics are often approximations of desired outcomes, and any discrepancy, however small, can have unintended effects. Every time we define "good answers" with numbers, we risk such approximations—and occasionally, a goblin might slip through.