Model Garage

Open the hood on neural networks.

Component-level model surgery, analysis, and composition. Extract attention heads, swap FFN layers between models, inject capability blades, and build hybrid architectures — all from a beautiful CLI or Python API.

Get Started View on GitHub

---

• Extract

  ---

  Pull real nn.Module components from any supported transformer. Attention heads, FFN layers, embeddings — ready to test in isolation.

  Extraction guide

• Inject

  ---

  Insert custom processing between any two layers. Scale activations, inject adapters, add capability blades — without modifying the model.

  Injection guide

• Analyze

  ---

  Capture hidden states, measure entropy, sparsity, and activation patterns. Understand what each layer actually does.

  Analysis guide

• Compose

  ---

  Register models, compare architectures, and find compatible parts. Build hybrid models from proven components.

  Composition guide

---

Quick Example

CLIPython

# Inspect a model's architecture
garage open gpt2

# Extract attention from layer 6
garage extract gpt2 --layer 6 --component self_attention

# Compare two models for compatible parts
garage compare gpt2 distilgpt2

# Analyze activations across layers
garage analyze gpt2 --prompt "The meaning of life is"

from model_garage import ModelLoader, ModelRegistry

# Load and decompose
loader = ModelLoader()
model, tokenizer, info = loader.load("gpt2")

registry = ModelRegistry()
spec = registry.register("gpt2", model)

# See all parts
for name, part in spec.parts.items():
    print(f"{name}: {part.part_type.value} [{part.input_dim}→{part.output_dim}]")

---

Supported Architectures

70+ models across 18 vendors with full surgery support.

Family	Models	Capabilities
GPT-2	gpt2, gpt2-medium, gpt2-large, gpt2-xl, distilgpt2	Extract, inject, analyze, compose
Llama	Llama-2-7b, Llama-3-8b, TinyLlama, CodeLlama	Extract, inject, analyze, compose
Phi	Phi-2, Phi-3.5, Phi-4, Phi-4-reasoning, MediPhi	Extract, inject, analyze, compose
Phi-MoE	Phi-3.5-MoE, Phi-mini-MoE, Phi-tiny-MoE	Extract, inject, MoE routing, blade injection
Mistral	Mistral-7B, Mixtral-8x7B	Extract, inject, analyze, compose
Gemma	Gemma-2b/7b, Gemma-3, FunctionGemma, MedGemma	Extract, inject, analyze, compose
Qwen	Qwen-1.5, Qwen-2, Kimi-K2.5	Extract, inject, analyze, compose
BERT	bert-base/large, distilbert, MiniLM, mpnet	Extraction, analysis
Protein	ESM2 (8M to 3B)	Extraction, analysis
BitNet	bitnet-b1.58-2B-4T	Extraction, analysis

---

Backed by Research

Model Garage is validated through three peer-reviewed papers with quantified results.

• Blades: Capability Transfer

  ---

  Hidden state injection achieves +14.2% accuracy on medical reasoning. 7 validated principles for capability transfer.

  Read more

• MoE Router Miscalibration

  ---

  Learned MoE routers show rho = 0.069 between routing probability and expert quality. 482/896 combinations show significant specialization — yet routers ignore it.

  Read more

• Sparse Pathways

  ---

  FFN neurons show domain specialization with r=0.999 scale correlation. 2-4x potential compute reduction via negative neuron selection.

  Read more