DEVİMDEVİM
← Back

devim-model · update

336M Gate-B: capacity effect not established; architecture-design gate opened

The matched 336M Gate-B evaluation is complete. Scale improved FORM and positive-control readouts but did not establish the preregistered capacity effect.

Status PublicPublished 1 min read210 wordsv1.1
ListenNot supported in this browser

Result

The 336M Gate-B evaluation is complete. Training had already reached the frozen target of 36,755 optimizer steps and 1,200,012,349 cumulative supervised tokens.

Matched Gate-B results:

  • breadth macro: 110M 0.3364583 → 336M 0.4333333
  • breadth delta: +0.096875; preregistered minimum +0.20
  • minimum 336M breadth family: 0.15; preregistered minimum 0.45
  • FORM EOS: 0.8333333 → 0.96875
  • FORM repetition: 0.1666667 → 0.0416667
  • V1.8 macro: 0.44125 → 0.44125
  • V1.8 competencies above chance: 4 → 4
  • positive-control macro: 0.7142857 → 0.8571429

Gate decision

Formal decision:

GATE_B_CAPACITY_EFFECT_NOT_ESTABLISHED_OPEN_ARCHITECTURE_DESIGN_GATE

capacity_effect_supported: false

The 336M model improved several readouts, especially FORM and positive controls, but did not satisfy the preregistered breadth requirements. The experiment therefore does not support the claim that the larger model established the target capacity effect.

What this does not mean

This decision does not mean that Transformers cannot work, that scale has no value, or that an alternative architecture is already superior. Several measurements genuinely improved; the target broad-transfer/capability effect simply was not established by scaling alone.

What changes now

The next research gate is no longer to add parameters by default. The architecture-design gate is open: preserve the data, evaluator and compute discipline, then test simpler alternative explanations and controlled architectural interventions against matched baselines.

PUBLICATION RECORDdevim-336m-gate-b-status
Version
v1.1
Evidence
2
Source
Git · en/research/336m-gate-b-training-complete.md

DEVİM publications preserve revision history and distinguish public evidence from internal work.

current state336MGate Barchitecture design