Skip to content

Research

Model Garage is validated through three peer-reviewed research papers demonstrating its capabilities on real problems. Each paper uses Model Garage modules directly and reports quantified, reproducible results.

Papers

  • Blades: Compositional Capability Enhancement Through Hidden State Injection

    Hidden state injection between specialized models achieves +14.2% accuracy on medical reasoning tasks. Establishes 7 validated principles for capability transfer including the N-4 layer rule and same-domain synergy (+27.8%).

    Modules used: extract, inject, snapshot, core.hooks

    Full summary | PDF

  • Learned Routers Don't Learn: Expert Miscalibration in MoE Models

    Per-layer expert isolation reveals that learned MoE routers show Spearman rho = 0.069 between routing probability and expert quality. 482/896 expert-layer-domain combinations show statistically significant specialization, yet the router ignores it.

    Modules used: analyze, core.hooks, registry

    Full summary | PDF

  • Sparse Pathways: Domain-Aware Neuron Routing for Efficient Inference

    FFN neurons exhibit strong domain specialization (~50% in late layers), with r=0.999 correlation between model scale and specialization degree. Demonstrates 2-4x potential compute reduction via negative neuron selection.

    Modules used: analyze, snapshot, core.hooks

    Full summary | PDF

Citation

If you use these findings or Model Garage in your research:

@software{model_garage,
  title = {Model Garage: Component-Level Neural Network Surgery},
  author = {Model Garage Contributors},
  year = {2026},
  url = {https://github.com/Lumi-node/model-garage},
  license = {Apache-2.0}
}

Pretrained Artifacts

The research directory includes pretrained components ready for use: