Research
Programs, not projects.
Each program is chosen to stay relevant across model generations. The questions are foundational, the methods are careful, and everything we publish is meant to be reproduced.
ARC, the Autonomous Recovery Controller
A self-healing AI infrastructure framework for PyTorch.
Training a large model is still a fragile process. Loss spikes, numerical faults, and silent divergence can waste days of compute before anyone notices. ARC watches training dynamics continuously, predicts instability before it becomes catastrophic, and recovers the run on its own.
The program explores reliable optimization, failure prediction, numerical robustness, checkpoint intelligence, and what it takes to make ML infrastructure genuinely dependable.
pyarc.pages.devScale-Dependent Structural Mimicry in Distilled Reasoning Models
Do distilled models reason, or do they imitate the shape of reasoning?
This work introduces Causal Trajectory Attribution, a method for studying reasoning traces in distilled language models. It asks whether the reasoning patterns we commonly observe reflect genuine causal reasoning or structural habits inherited from larger teacher models.
The answer matters for how much we should trust chain-of-thought as evidence of understanding, and for how reasoning ability transfers when models are compressed.
How Neural Collapse Diverges in Self-Supervised and Supervised Learning
Watching representation geometry form, frame by frame.
This project studies Neural Collapse at high temporal resolution and compares supervised with self-supervised learning. The early results show that representation geometry, collapse dynamics, and downstream transfer performance are more nuanced than previously believed.
Understanding when and why representations collapse gives us better tools for diagnosing training, and better theory for why transfer works at all.
Lagrange Paralysis: Decoding Deadlocks, Contractive Geometry, and the Price of Escaping Them
The model is torn between two tokens. Breaking the tie costs more than leaving it.
Sometimes a model reaches a next-token decision with its top two candidates almost exactly level. We call that state Lagrange Paralysis, and it is cheap to detect. Breaking the tie looks like the obvious fix, so we tried nine ways of doing it across four open models.
Only one method pulled the model off its greedy path, and it was the one that wrecked downstream reasoning, halving accuracy on math benchmarks. The geometry hints at why: hidden states at tied steps carry about half the effective rank of confident ones. Noticing these moments is easy. Acting on them, so far, costs more than it returns.
Interested in collaborating on any of these programs?
Get in touch