If a local GDN model lost to a hosted API in your evaluation, check the local runtime revision before turning that result into a routing rule. llama.cpp’s b10829 release describes a normalization correction that makes an affected older build a questionable baseline for the corrected one—not evidence that either deployment now wins.
What changed
The b10829 release notes, published September 6, describe a fix to gated delta net (GDN) query/key normalization. The reference form from flash-linear-attention is:
x * rsqrt(sum(x*x) + eps)
The previous GDN call sites instead used:
x / max(sqrt(sum(x*x)), eps)
Those put epsilon in different places: inside the square root versus in a clamp on the denominator. The release explanation says the clamp did not engage at the magnitudes involved, leaving those calculations effectively without the reference’s epsilon contribution.
This is a call-site correction, not a claim that every L2 normalization operation was broken. The notes explicitly say ggml_l2_norm itself remains correct and unchanged, as does its original rwkv7-base caller. Epsilon still comes from the checkpoint.
My inference: an evaluation on an affected GDN build mixes model behavior with the old implementation. The release explanation alone does not establish a task-quality improvement, its size, or anything about a hosted provider’s backend.
A useful rerun before changing routes
For a team deciding which requests stay local and which go to an API, I would use three evaluation columns:
| Column | What to preserve | What it can tell you |
|---|---|---|
| Old local build | Exact runtime revision and saved per-case outputs | What the original routing decision actually used |
| Corrected local build | Same weights, quantization, prompts, context settings, sampling and scoring | Whether the local result changes under the corrected runtime |
| Hosted API route | Requested model/version where available, request settings, timestamp and per-case outputs | How that endpoint compares on the same tasks during this evaluation |
This is a proposed protocol, not a reported benchmark. Keep the old and corrected local runs as tightly matched as possible; record other build or serving differences rather than attributing every output change to this one fix. If the API does not expose a pinnable backend revision, mark that comparison as time-bound.
Define “accepted output” before running the task set—for example, valid structured output and the expected extracted fields, not merely a successful HTTP response. Report acceptance counts with denominators and inspect which individual cases changed. Repeat stochastic cases rather than treating one changed completion as a stable gain. Record errors, latency and cost separately so that a quality change does not silently stand in for an operational advantage.
The decision boundary is narrow: rerun an affected local baseline before revising the routing policy. Do not carry its old score forward as the corrected runtime’s score. If acceptance is unchanged, the fix alone is no reason to move traffic. If it changes, re-evaluate the quality/latency/cost tradeoff against the hosted sample.
Even then, different weights, quantization or serving settings make this a deployment comparison—not proof that one model architecture is better. The useful outcome is a defensible routing decision, not a winner inferred from a release note.