SANGAMAM
Composable Language Model Architecture with Formal Interface Contracts
What It Is
An LLM architecture for uploadable machine learning with architectural composability guarantees. Any lab trains a specialist above the published frozen Foundation Spine; the SANGAMAM Composability Interface (SCI) is the formal contract that guarantees specialists compose correctly without central coordination. Builds on the empirical fusion-gain proof from KALAVAI — but makes composability *designed*, not accidental.
The Problem
Fine-tuning a single model breaks other domains. Naive merging is unprincipled. Even KALAVAI proves *that* fusion works — not *that any two arbitrary checkpoints will compose*. SANGAMAM asks: can composability be a designed architectural guarantee, not a property you hope for after the fact?
Approach
- Foundation Spine — a single frozen transformer backbone published to HuggingFace, versioned, the universal interface (d_model=1024 at 410M)
- SCI v1.0 spec — 8-check programmatic compliance contract (d_model, vocab, RoPE base, norm type, residual type, activation, spine version) every specialist must pass before publishing
- Specialist Block training pipeline — independent training above the frozen spine, MUST pass SCI compliance before release
- Composition Layer — lightweight learned top-k router (k=2), trained in ~500 steps; supports incremental assembly (add specialists post-hoc without retraining the router)
- Fusibility score — pre-training CKA-based metric predicting downstream composability quality before compute is spent
- E1–E9 experiment suite including a controlled SANGAMAM-vs-KALAVAI head-to-head at matched compute
Results
- Phase 0 scaffolding complete: SCI spec + compliance harness + 125M dev spine dry run on RTX 5090
- Phase 1 target: 410M spine published to HuggingFace (RunPod A100, ~$150–300 budget)
- Four formal claims framed for NeurIPS: composability contract, designed > accidental composability, pre-training fusibility metric, incremental assembly protocol