devim-model · research
DEVİM research timeline
From the 43M reference and 110M diagnosis through FORM/CONTENT separation to the final 336M Gate-B decision.
This is not a success-only timeline
DEVİM has not progressed as a simple story of "the model got bigger and better." At several stages, the evidence forced us to change the diagnosis. This timeline includes failed experiments because those failures changed what we tested next.
1 — Small-scale reference
An approximately 43M-parameter model was retained as a historical architecture-family reference. Its role was not to become the final system, but to help test whether later changes were truly explained by scale.
2 — 110M natural-only training
The approximately 110M-parameter plain causal Transformer reached about 1.2B supervised tokens by Gate B. Held-out cross-entropy improved substantially, while the frozen V1.8 capability score barely moved.
That was the first major break in the story: better language modeling was not the same thing as measured relational capability.
3 — The interface problem became visible
The model also failed at the response surface: EOS was low and repetition was severe. We therefore had to separate "the model does not know" from "the model never learned the answer register."
4 — QA v0.1: partial repair, new problem
A small diagnostic child was trained on mechanically derived grounded QA examples. EOS and repetition improved, but frozen success bars were not cleared and held-out CE regressed. The child was rejected as a future parent lineage.
5 — QA v0.2: evaluator copy confound
A stronger control showed that a large part of the task could be solved by trivial copying. Extractive QA scores therefore could not be treated as direct evidence of relational understanding.
This made the evaluator itself part of the research problem.
6 — FORM Rescue
At the same parameter scale, more balanced response-register supervision produced much better termination and repetition behavior. The original speaking/interface failure therefore could not be explained by parameter count alone.
But better FORM was not treated as solved CONTENT.
7 — CONTENT and relation binding
Controlled curricula produced strong local signals in relation binding and abstention, but transfer to fresh surfaces and direct generation remained weak. Adversarial review described the evidence as shortcut-dependent learning with weak transfer.
The research question changed from "can the model learn this task?" to "can it carry the learned relation to a new surface?"
8 — Grounded Turkish data
The response to shortcuts and evaluator weaknesses was not simply "generate more data." A grounded Turkish supervision program was built around provenance, sequestered evaluation material, and explicit answer-present / answer-absent structure.
9 — 336M matched scale probe
Instead of inferring "110M is too small," that hypothesis was tested under a matched plain-Transformer setup. The approximately 336M model improved several Gate-A readouts but did not pass the preregistered capacity gate.
Formal Gate-A decision: GATE_A_NOT_ESTABLISHED_CONTINUE_GATE_B.
10 — Gate B: scale alone did not establish the capacity effect
336M Gate-B training reached the frozen target of 36,755 steps / 1,200,012,349 supervised tokens, and the matched evaluation was completed.
The 336M breadth macro reached 0.4333333, compared with 0.3364583 for the matched 110M condition. But the delta was only +0.096875, below the preregistered +0.20 requirement. The minimum breadth family was 0.15, below the required 0.45.
At the same time, FORM EOS improved to 0.96875 and positive-control macro to 0.8571429. Scale genuinely improved several readouts, but it did not establish the target broad capacity effect.
Formal Gate-B decision: GATE_B_CAPACITY_EFFECT_NOT_ESTABLISHED_OPEN_ARCHITECTURE_DESIGN_GATE.
11 — Architecture-design gate
This does not mean "Transformers failed." The next research question is which controlled architecture or objective change can actually reduce weak transfer when tested against matched baselines, rather than simply adding more parameters.
Parallel track — Computational Dialectics
In parallel, but independently, Computational Dialectics is trying to make revision and relational mechanisms explicit and falsifiable. It does not block DEVİM training and only becomes an integration candidate if it earns evidence against matched ordinary baselines.
The current research question
The question is no longer simply "is a larger model better?" It is: which data, objective, or architectural change can establish the transfer/capability effect that scale alone did not?
- Version
- v1.1
- Evidence
- 0
- Source
- Git · en/research/research-timeline.md
DEVİM publications preserve revision history and distinguish public evidence from internal work.