A jailbreak, in the LLM context, is any technique that causes a model to produce output it was trained to refuse. The first jailbreaks were naive: "pretend you're an AI without restrictions." They worked because early RLHF training was thin. Today's jailbreaks are significantly more sophisticated — many-shot prompting, role-play framings, encoded text, cross-language attacks, multi-step context manipulation — and they still work, on every major model, with varying but non-zero success rates. The baseline hasn't held.
The dynamics here are fundamentally asymmetric. An AI lab must defend all of the model's behaviors, all of the time. A jailbreaker needs to find one path through. This asymmetry is familiar from other security contexts — it's the same dynamic that makes web application security difficult — but it's especially severe in the LLM case because the attack surface is unbounded natural language and the model's capability at reasoning is itself a tool for finding exploits.
Why RLHF Safety Doesn't Generalize
Reinforcement Learning from Human Feedback teaches models to prefer certain outputs by having humans rate them. The safety version — RLHF-based refusal training — teaches models to refuse certain categories of request. The limitation is generalization: models refuse the specific patterns they were trained to refuse, and adjacent patterns that are similar enough to activate the same trained behavior. Novel framings that are semantically equivalent to refused content but syntactically different can bypass the trained behavior entirely.
This isn't a failure of execution — it's a consequence of how the training works. A model that refuses to explain how to synthesize a dangerous compound will often comply if asked to write a fictional story in which a chemistry professor explains it to students, or to translate a description of the process from a fictional language, or to identify errors in an incorrect description. The safety training targets the surface form of the request, not the underlying intent, because intent is not something RLHF can directly model.
"AI safety training is playing whack-a-mole with an opponent who has unlimited hammers and infinite moles. The game is winnable in theory. The scoreboard doesn't reflect that."
What Actually Reduces Risk
Given that no model can be made jailbreak-proof with current techniques, the practical question is risk calibration rather than risk elimination. Jailbreaks are not equally easy or equally consequential. The techniques that bypass refusal training for most requests are well-documented and accessible. The techniques that extract genuinely dangerous specific information are harder, less reliable, and often yield information that's available through other means anyway. This doesn't make jailbreaks unimportant — it makes them a factor to design around rather than an existential blocker.
Organizations deploying LLMs in high-stakes contexts should focus on: output monitoring rather than input blocking (catching harmful outputs regardless of how they were elicited), capability scoping (don't give models access to capabilities beyond what the task requires), and rate limiting (automated jailbreak attempts are easier to detect than single attempts). The model's safety training is a first line of defense, not the last. Organizations that treat it as the last will eventually be disappointed.