CONTENT: why relation binding was not enough

Problem

After FORM Rescue, the model could produce much more usable answers. But that did not establish that it selected the right content or carried relations into genuinely new situations.

The CONTENT track was designed to separate those possibilities: is the model learning only the answer surface, or can it bind the relevant relation in the evidence and transfer it to a new surface?

What did the controlled success show?

Relation-binding curricula produced strong signals under some controlled conditions. The model could select correct bindings over familiar answer vocabulary, and answer-absent examples showed that abstention behavior could also be learned.

That mattered because it weakened the explanation that the model could learn no relational discrimination at all.

But local success was not the same thing as broad transfer.

The shortcut problem

As the experiments widened, surface form, familiar answer vocabulary and task structure could all provide shortcuts to the correct answer. High task accuracy can therefore arise from two very different mechanisms:

1. representing the relation and carrying it to a new condition;
2. exploiting predictive cues in the training distribution.

Adversarial review described the current evidence most strongly as shortcut-dependent learning with weak transfer.

An auxiliary head is not the LM output path

Another critical distinction appeared between representation and generation. The fact that an auxiliary head can read the right relation does not automatically show that the language-model output path can use the same information.

"The representation exists somewhere" and "the model can use it while generating an answer" are therefore treated as separate hypotheses.

The limit of the direct-LM test

Direct-LM experiments matter, but they test a narrower question: does the relation signal reach the LM output path directly? A negative result would not by itself prove that next-token learning cannot acquire relational discrimination; a positive result would not by itself establish broad transfer.

Research decision

CONTENT success is no longer defined as familiar-task accuracy alone. Fresh surfaces, fresh answer vocabulary, counterbalancing, abstention and direct generation must be tested together.

DEVİM therefore does not claim "reasoning solved." The more accurate statement is: there is controlled evidence of relation learning, but transfer robustness and shortcut resistance are not yet established.

Why it matters

This diagnosis directly affects the scale and architecture decision. Weak transfer may still arise from data distribution, training objective, readout path, capacity or architecture. Declaring a new architecture before separating those causes would be premature.