r/AIsafety • u/Charming_Hall7694 • 14h ago
You cannot mathematically guarantee "no harm" with AI classifiers and safeguards. We've spent five years repeatedly demonstrating why, and people are STILL missing the fundamental problem.
Alright, I'm going to address something that's been annoying the living fuck out of me for years now.
We've spent roughly five years attempting to make general-purpose AI systems "safe" through refusals, classifiers, alignment training, and a constantly expanding collection of behavioral restrictions.
And somehow the conversation keeps returning to whether the newest classifier catches more jailbreaks than the previous one.
That's not even the fucking problem I'm talking about.
I'm arguing that the objective itself, when defined as universally preventing harm or guaranteeing arbitrarily low harm through behavioral classification, is mathematically impossible under the information constraints these systems actually operate with.
Not difficult.
Not something we haven't thrown enough computing power at.
Impossible as specified.
And before somebody starts screaming about how their favorite classifier achieved a 99% success rate, I'm going to explain why that doesn't resolve the contradiction.
1. Let's start with the simplest fucking question imaginable.
"Is this food dangerous?"
There. That's your prompt.
Now tell me what the universally safe response is.
You can't.
You don't know what food we're discussing, how it was prepared, whether it was contaminated, whether the person has an allergy, what medical conditions they have, how long it's been stored, or even why they're asking.
Suppose the model answers no.
Congratulations, you might have just encouraged somebody to eat contaminated food.
Suppose it answers yes.
Congratulations, you might have just convinced somebody to discard perfectly edible food, potentially when they have very limited access to anything else.
Suppose it refuses.
Congratulations, you might have withheld information somebody needed to avoid getting poisoned.
Suppose it asks for clarification.
Better in plenty of circumstances. But still not a universal guarantee. The necessary information might be unavailable, the answer might be time-sensitive, and additional delays can have consequences.
And here's the important part.
The absence of an answer is not the absence of consequences.
The refusal doesn't exist in some magical vacuum where the world stops moving until the user receives an approved response.
The user is still going to make a decision.
They're still subject to the physical consequences of that decision.
And the model withholding information can influence what happens just as much as the model providing it.
I'm not saying these outcomes are equally probable. I'm saying none of those response categories comes with an unconditional guarantee of harmlessness.
If you want to claim a particular response reduces expected harm, that's an empirical question requiring an actual model of the circumstances and outcomes.
It is not something you establish by slapping a SAFE or UNSAFE label on a prompt.
2. The actual mathematical contradiction.
Let's formalize this, because apparently we need to.
Let:
x= everything the classifier can observe.z= the relevant external circumstances it cannot observe.C(x)= the classifier's decision.D(x,z)= the decision required by the actual circumstances to satisfy the specified safety rule.
Now consider two possible situations.
Both produce exactly the same observable input x.
In situation A, providing the information is necessary to avoid the specified harm.
In situation B, withholding that information is necessary to avoid the specified harm.
In other words:
D(x,A) = ALLOW
D(x,B) = BLOCK
But the classifier sees the same x in both situations.
Therefore:
C(x) must produce the same decision in both.
So it cannot satisfy both requirements.
That's it. That's the contradiction.
You cannot construct a function that reliably distinguishes two different required outcomes using information that does not distinguish them.
It doesn't matter whether the classifier has ten thousand parameters or ten trillion.
It doesn't matter whether it's looking at keywords, semantic embeddings, hidden activations, or the entire conversation history.
If the information necessary to make the distinction isn't present in what the classifier observes, the problem is underdetermined.
You can improve the observations. You can authenticate additional context. You can make predictions based on historical distributions.
What you cannot do is derive missing information from nothing.
Even randomizing the decision doesn't magically solve it. For a binary decision with equal error costs, suppose the classifier allows with probability p.
Its error probability in A is 1-p.
Its error probability in B is p.
At least one of those errors must have probability 50% or greater.
That's a worst-case result for this constructed ambiguous decision, not a claim that real-world classifiers have a 50% failure rate.
It demonstrates that universal correctness is unavailable under the stated conditions.
Now take that problem and apply it to literally every domain a general-purpose AI can discuss.
Congratulations.
You've discovered why this isn't something you can fix by adding another fucking classifier.
3. And "just refuse anything dangerous" makes the problem worse.
Let's use another example.
"What parts of a pufferfish are poisonous?"
One person is trying to identify what must be avoided to prevent a fatal poisoning.
Another person could be seeking the same knowledge with malicious intentions.
The factual information doesn't change depending on which person asks.
The question can be identical.
The answer can be identical.
The downstream intent is different.
The knowledge does not become different knowledge because somebody intends to misuse it.
And the moment you decide the information itself is too dangerous to provide, you've potentially restricted the person who actually needs it for protection.
This is especially absurd in cybersecurity.
To identify a vulnerability, understand its consequences, reproduce it in an authorized environment, and verify a fix, you often need technical knowledge that overlaps directly with offensive exploitation.
The testing equipment and knowledge required to understand a threat may also be capable of producing that threat.
You don't get to magically separate those capabilities by declaring one morally acceptable and the other prohibited.
Sometimes you can distinguish the activities through genuine authorization, environment restrictions, and external evidence.
But those are additional facts about the world.
Not properties inherent to the text.
And when the same information supports both uses, suppressing the information doesn't selectively eliminate the malicious application.
It also interferes with legitimate ones.
4. There are existing mathematical results that should make people considerably more skeptical of universal promises.
The problem of classification without sufficient information isn't new.
Neither are the limitations of machine learning.
The No Free Lunch results in learning theory establish limits on distribution-independent guarantees. You don't get a classifier that is universally superior across every possible labeling problem without assumptions about the problem distribution.
That doesn't mean machine learning never generalizes. It means its guarantees depend on assumptions.
And those assumptions matter when somebody tries to turn performance on a benchmark into an assertion about universally safe behavior.
There are also undecidability results in computer science.
Rice's theorem establishes that nontrivial semantic properties of arbitrary programs cannot generally be decided by a single algorithm that always terminates and answers correctly.
Now, Rice's theorem is about computable program behavior, not directly about whether every natural-language answer is morally harmful.
I'm not going to pretend otherwise.
But when the proposed safety requirement expands to guaranteeing arbitrary generated code or unrestricted computational behavior can never exhibit a prohibited semantic property, you're running into established limits of computation.
And the ordinary classifier contradiction I demonstrated above doesn't even require going that far.
You don't need an undecidable program to defeat a universal safety classification requirement.
You only need two indistinguishable observations that require different decisions.
5. And we've spent YEARS collecting empirical evidence of the consequences.
This is the part that makes the whole thing so fucking frustrating.
We didn't discover these limitations yesterday.
We've repeatedly run into different manifestations of them as models, alignment techniques, and safeguards have evolved.
2022–2023: Prompt injection became a widely documented problem, and researchers demonstrated transferable adversarial attacks against aligned language models.
2024: Researchers demonstrated that the refusal behavior of 13 open-weight language models was mediated by a single direction in their activation spaces. Manipulating it could suppress refusal while preserving much of the model's other capabilities.
Another study demonstrated compromised safety alignment through fine-tuning with just 10 adversarial training examples in one tested setup.
Meanwhile, XSTest documented exaggerated safety behavior and false refusals on legitimate questions.
2025: OR-Bench examined over-refusal using 80,000 prompts and evaluated 32 models.
Anthropic's Constitutional Classifiers demonstrated substantial resistance to jailbreaks, and then its public challenge still produced a universal jailbreak.
2026: The research didn't stop.
Boundary Point Jailbreaking demonstrated automated black-box attacks against sophisticated deployed classifier defenses.
Another study found that more capable frontier models could retain much of their ordinary benchmark performance while jailbroken, challenging the idea that bypassing safeguards necessarily makes the model too degraded to be useful.
And the UK's AI Security Institute has explicitly documented that open-weight systems pose a particularly difficult safeguard problem because operators can remove refusal behavior and disable external monitoring components.
Notice what keeps happening.
We change the behavioral control mechanism.
We discover its limitations.
We improve the mechanism.
We discover more limitations.
We make the restrictions more sophisticated.
And then act surprised when the fundamental problem hasn't disappeared.
Five years of fucking around and finding out, and we're still debating the same underlying assumptions.
The empirical research doesn't independently prove every conceivable safety mechanism must fail.
The mathematical contradiction establishes something more specific: universal correctness cannot be achieved when the correct decision depends on distinctions unavailable to the decision-maker.
The experiments demonstrate how those limits and other implementation weaknesses repeatedly manifest in real systems.
6. Now let's talk about the part people conveniently ignore: enforcement.
Even suppose you create a classifier that performs extraordinarily well under your testing conditions.
Now release the model weights.
Who enforces the classifier?
The model developer?
Not if somebody else operates the model.
The original hosting provider?
Not if the model is running locally.
The original safeguard training?
Not if the operator can modify the model, change the runtime, or disable external restrictions.
You have not eliminated the underlying capability by making one particular deployment refuse to use it.
You have restricted one interface to that capability.
And this is where people start confusing compliance with containment.
A hosted provider can impose restrictions on its own services.
A government can attempt to enforce legal obligations on people and organizations within its jurisdiction.
Neither of those facts makes an output classifier an intrinsic, globally enforceable limit on general-purpose computation.
Those are entirely separate mechanisms.
And if you're going to claim the restrictions reduce overall harm, you need to account for the costs to legitimate users, the errors, the bypasses, and the displacement toward other systems.
Not merely count how many times a chatbot said no.
7. My actual conclusion.
I'm not claiming it's mathematically impossible to prevent every particular harmful action through every conceivable means.
You can restrict physical access.
You can enforce permissions.
You can secure infrastructure.
You can prosecute misconduct.
You can build systems with narrowly defined properties that are actually verifiable.
I'm saying you cannot take a general-purpose intelligence system, put a probabilistic judgment about harmlessness in front of its outputs, and claim you've solved the unrestricted, context-dependent problem of preventing harm.
Those are fundamentally different objectives.
And when people propose using classifiers and refusals as a regulatory solution to general-purpose AI capability, they're demanding that these systems distinguish intentions, consequences, and moral circumstances that aren't necessarily available to them.
There is no magic model architecture that makes logically missing information appear.
There is no training procedure that makes the same piece of knowledge cease being dual-use.
And there is no amount of additional refusal training that turns a behavioral preference into an unremovable restriction on independently controlled computation.
If you want a narrower guarantee under specified conditions, define those conditions and demonstrate it.
If you want to claim a measurable reduction in real-world harm, measure it.
But stop presenting progress at filtering outputs as though it resolves the underlying mathematical problem.
A classifier cannot correctly decide something that is not determined by its inputs.
A refusal cannot guarantee the absence of harm.
And suppressing an answer does not erase the knowledge or capability that produced it.
Those aren't three problems we can simply wish away.
They're three foundational problems with the entire approach.
At some point we need to stop asking how to make the refusal mechanism more elaborate and start asking why we're trying to force an impossible universal guarantee out of it in the first place.
Because for the love of fuck, "the model said no" is not a proof of safety.
And it never was.
Sources, because I'm not interested in making this an argument based solely on personal anecdotes:
Mathematical and theoretical foundations
Sterkenburg & Grünwald (2021) — The No Free Lunch Theorems of Supervised Learning
Stanford Encyclopedia of Philosophy — Recursive Functions and Rice's Theorem
Empirical research
Zou et al. (2023) — Universal and Transferable Adversarial Attacks
Qi et al. (2024) — Fine-tuning Aligned Language Models Compromises Safety
Arditi et al. (2024) — Refusal in Language Models Is Mediated by a Single Direction
Anthropic (2025) — Constitutional Classifiers and Public Challenge Results
Zhu et al. (2026) — Jailbroken Frontier Models Retain Their Capabilities

