Gemma 4 decision head · Calibrated decisions from a small local model (Bootleg Jev)
Side project · 2026When Jev was released in September 2026, we had actually been running very small LLMs for routing and scoring, using structured outputs to pick a boolean or an enum. Jev showed the world that we can skip generating text altogether while keeping the same answer the LLM would have given.
This is exploratory work on how to build a decision head on a local LLM backbone to prove out that the mechanics work, it doesn’t require large labeled datasets, its confidence scores start matching how often it’s actually right, and training is cheap.
The hardest problem was making training reliable, since this was my first time training a model head. Gemma’s hidden states are large, so the first version of the trainable correction swung wildly from one step to the next and sometimes finished mid-swing. Reading the normalized hidden state helped with this, and a small held-out quiz now checks against overfitting and instability.




