Edit One Operator, Edit Every Depth

Two posts ago, a network was a list you could edit. Its whole trick was that knowledge had an address: a class was a handful of rows you could point at, so teaching was pasting rows in, forgetting was deleting them, and you could diff the weights afterward to prove nothing else moved. Then the last post took the stack of layers those rows lived in and melted it down: one shared operator, applied over and over until the state stops moving, the answer a fixed point rather than a final layer. It was a good trade. But look at what it did to the address book.

In the list network, a pasted row entered the score once, as one more term in a sum, and the proof that old classes could not feel it was a single line. Here there is no “once.” Whatever you paste into the one shared operator is applied at every depth, and its output is fed back into its own input, around and around until the equation settles. An edit does not add a term; it changes the map whose fixed point is the answer, and every fixed point is free to drift. Worse, the whole argument that “iterate until it stops” returns one well-defined answer rested on a certificate, the measured slope of the operator staying below one, and nothing says a paste respects it. So the question this post exists to answer: did melting the stack melt the editability too?

The short version: it survives, but it splits, and the guarantees follow the kind of row you touch, not the wish. The rest of this post is finding the split and paying its bills. Every number is from a real run (scripts/yat_deq_edit.py), and every panel below runs the real operator live in your browser.

Where would an edit even go?#

The machine under edit is the one the last post built, retrained with a third class in the mix, so one paragraph of recap and no more. A bank of $M = 24$ shared prototypes and one mixing matrix define a single operator on a 32-dimensional state,

iterated from $z = 0$ until it stops at $z^\* = F(z^\*;x)$, then read out linearly, $\hat y = C z^\* + c$. Ours is trained on three classes, two interleaving moons and one Gaussian blob, and lands at 97.9%, 98.1%, and 100% per-class recall from 1,733 parameters shared across all depth. Its contraction certificate, the measured spectral norm $\lVert J_F\rVert_2$ at the fixed points of the test set, sits at mean 0.629, max 0.904: comfortably below one, a single basin, one answer from anywhere.

Now suppose a fourth class shows up, a second blob the network has never seen, and you refuse to train. In the list network the move was obvious because everything was rows. Here the anatomy offers exactly two places to put new rows, and the entire post hangs on the difference between them. There are the rows that read the settled state: the readout is a matrix, and a matrix accepts new rows. And there are the rows that steer the state: the shared bank $W$ inside $F$ itself, the wells the state rolls through on its way to rest. Editing the first leaves the equation alone. Editing the second edits the equation, at every depth at once.

Teach it without touching the flow#

The timid door first, because it is the one that comes with a proof. To teach the new class, take eight examples of it, drop each into the frozen network, and let the operator do what it always does: iterate them to their equilibria. Those eight settled states $c_1, \dots, c_8$ are where the flow already parks the new class. Paste them in as readout rows, scored through the very same Yat kernel the operator uses, and give the new class the score

with $\alpha$ one calibration constant that puts the kernel on the scale the trained logits speak at. No gradient steps. The new class is recognized at 85.8% recall.

And the old classes? Here the proof from the editing post survives verbatim, because $F$ was never touched. Same operator, same fixed points, same linear readout of them: the old scores are not approximately preserved, they are the identical computation. The run asserts it: max $|\Delta\,\text{old logit}| = 0.0$, exactly, and zero old test points get captured by the new class. Teaching by readout is C2’s append-a-row, alive and well inside the recursion.

A new mass, but in whose landscape?#

Why does the timid edit feel timid? The physics the series has been carrying says it plainly. In the editing post a prototype was a mass and teaching dropped a new one into input space, where it dug its own well into a static landscape. The fixed-point post moved the landscape into state space and made the iteration the falling: the operator is a flow, the contraction means one basin, and the answer is the rest point at the bottom.

The readout edit hangs a new instrument beside that landscape without touching it. The state rolls exactly where it always rolled; we just read its resting place against eight new reference points. That is why the invariance is exact and why the edit cannot break anything: a gauge cannot alter the thing it measures. But it also cannot help the thing it measures. The new class’s inputs settle wherever the old flow happens to drop them, and the flow was never carved for them. You can see the shortfall in two numbers from the run: the new class’s margin over the old scores is a thin 0.55, and its test inputs settle a mean distance of 0.159 from their nearest anchor, a loose cloud around the pasted rows rather than a tight one. Recall 85.8% is what reading an uncarved landscape buys.

To do better, the mass has to go into the landscape. Drop the eight anchors into the shared bank, $W \leftarrow [W; c_k]$, with matching mixing columns $\gamma\,\hat c_k$, and the operator itself now has wells at the new class: the flow bends toward them, at every depth, because every depth is the same operator. That is the strong door. One paste, present everywhere. And it is exactly the door with no proof behind it.

Through the strong door#

What could go wrong with deepening a well? In the list network, nothing: an appended unit added one bounded term to one class’s score, and the story ended there. In a recursion the pasted rows feed their own output back into their own input. Make the well too deep and the flow around it becomes steeper than the contraction allows; the slope of $F$ crosses one, and the moment it does, the whole foundation of the model, one basin, one fixed point, “iterate until it stops” returning one answer, is no longer guaranteed. The certificate is not paranoia. It is the load-bearing wall.

And the natural paste does break it. Choosing $\gamma$ so each anchor gets a healthy pre-activation bump, the raw dynamics edit sends the measured slope to $\lVert J_F\rVert_2$ max 1.438 over the test probes. Contraction broken, uniqueness void: the paste was an act of vandalism against the equation.

The repair is one number. The edit has a single gain knob, so bisect it: find the largest fraction of the raw $\gamma$ whose measured slope stays under a ceiling of 0.98, a hair of safety below one. The run lands on $0.462\,\gamma$, and the certificate reads mean 0.646, max 0.980. Certified again, by measurement rather than hope. And the tempered paste still does its job, spectacularly better than the readout alone: new-class recall goes 85.8% to 100%, the margin fattens from 0.55 to 3.83, and the settle distance to the nearest anchor tightens from 0.159 to 0.097. The flow now carries the new class’s inputs home instead of merely labeling where they stall.

What did that cost everyone else?#

The readout edit’s invariance was a proof. The dynamics edit’s cannot be, the map moved, so what remains is a measurement, and this is where the story earns its keep: over all 520 old-class test inputs, solve every fixed point under the base operator and under the edited one, and subtract.

The drift $\lVert \Delta z^\* \rVert$ has median 0.020 and 90th percentile 0.071, on settled states whose norms run around 3.7: for the typical old input, the basin bottom moved by about half a percent of the state’s scale. But the tail is real and deserves its numbers: 29 of 520 inputs moved more than 0.1, the worst by 1.47, and the score changes follow the same shape, median 0.029, max 1.47. Three predictions flipped, all near boundaries, taking the old classes from 97.9/98.1/100 to 96.8/97.6/100. Settle times barely notice: the old classes’ median goes 18 to 19 turns.

So say it precisely, because this is the difference between this post and its parent. The recursion downgraded the dynamics edit from provably untouched to measurably barely touched: a median drift of 0.02, a worst case of 1.47, three flips in 520. Nothing here is exact; what remains is that the disturbance is small, concentrated near the new class, and, unlike any fine-tuning run, fully auditable, because the same solver that computes the answer computes exactly what the edit did to it. If you need the proof back, the readout edit still has it.

And what did all of it buy, against the alternative of just training? One comparison settles it, and it closes the loop with the editing post’s incremental-learning result. There, teaching classes one at a time landed exactly where training on everything from scratch did, 79.4% equals 79.4%, because superposed fields cannot remember arrival order. Here the from-scratch four-class model reaches 98.75% overall; the incremental network, base plus constructed class, reaches 96.1% with the readout edit and 98.3% with the certified dynamics edit. Not equal this time, and the reason is plain: the from-scratch model’s flow was trained for all four classes, while ours had the fourth carved in by hand. Half a point is the price of never having taken a gradient step on the new class. The recursion turned an identity into a near-tie, which is the same downgrade the invariance just took, exactness traded for a small, measured gap.

The third door#

There is a tempting third door this whole time, and it is worth walking through it just to watch it slam. The old instinct says: edit a layer. The recursion even seems to allow it: apply the edited operator for the first few turns, then let the original take over, the equilibrium version of splicing a modified layer into a stack. What survives at the fixed point?

Nothing. Run the hybrid with the edit steering for the first 3, 10, or 30 turns and the base operator afterward, and the state lands back on the base equilibrium to within $3.3 \times 10^{-7}$, while the solver’s own noise floor, the discrepancy with no edited turns at all, is already $2.4 \times 10^{-7}$. Not weakened, erased. The reason is the same contraction that makes the model trustworthy: the base flow has one basin with one bottom, so the moment it takes over, it forgets everything the excursion did, geometrically fast. The certificate that guarantees the answer also guarantees amnesia toward any edit that is not still there at the end.

Which sharpens the title of this post into a statement about necessity. In an equilibrium model there is no such thing as editing a layer, because a fixed point only remembers what acts on it at every depth. Weight-tying does not merely permit the everywhere-edit, it is the only edit there is. One paste into the shared operator is automatically that everywhere-edit; anything less evaporates.

What a stack would charge for the same edit#

The evaporation result says an edit must exist at every depth to exist at all. The tied network gets that for free; what does it cost when the weights are not shared? The run trains the control: the same architecture with the tying switched off, twelve untied layers, which needs 18,981 parameters to the loop’s 1,733 before anyone edits anything.

Reading is cheap everywhere: pasting anchor rows onto the stack’s final state teaches the class at full recall with zero dynamics edits, the readout route again, depth-free in any architecture. But putting the class into the dynamics, the thing that carved the wells and fattened the margin above, has no single place to go. The paste must be repeated at each layer where the edit should exist: 512 numbers per layer, 6,144 to cover all twelve. Each pasted layer shoves old inputs’ final states further, from 0.46 with one layer to 3.86 with all twelve, well past the tied edit’s worst case of 1.47. And there is nothing to certify: a stack has no fixed point, so there is no contraction to re-check, no equivalent of the 0.980 receipt, no principled knob to bisect. You pay twelve times, disturb more, and get no certificate for any of it. One operator, one paste, one measurement is the entire tied bill.

Forgetting splits into two ledgers#

Now run the whole thing backwards, because unlearning was the editing post’s sharpest result, delete the rows and the class is provably gone, and the recursion does something more interesting than preserving it: it splits it in two.

Forget the class we just taught, and the old guarantee returns in full. Its knowledge sits in eight appended rows of $W$ and eight columns of $A$; slice them off and the run checks the result against the pre-edit operator: bit-for-bit identical, every array equal. What construction pasted in, deletion removes exactly, fixed points and certificate restored by the strongest possible argument, the weights are the same weights.

Now try to forget a class the network was trained on, and the address book fails for the first time. The blob class has one readout row, but its knowledge does not live there; gradient descent wove it into the shared bank that every class’s flow runs through. Mask the row and the surface result looks like exact unlearning: recall 1.00 to 0.00, and the other classes read literally the same numbers, 97.9/98.1 before, 97.9/98.6 after, the tiny gain being one moon point the blob had been capturing, handed back to its owner. But the flow still carries the class. Paste eight anchor rows, the same construction that taught class 3, and the “forgotten” class comes back at 100% recall, no training. It was unreadable, never erased.

That is the boundary, and it is the finding of this post: inside a fixed point, what you constructed, you can delete exactly; what you trained, you can only silence. True erasure of trained knowledge would mean editing the dynamics it is woven into, and the evaporation section already told you such an edit must reach every depth, which is retraining by another name. The editing post’s exact unlearning was never a property of the architecture alone; it was a property of knowledge stored as rows. The equilibrium model keeps the guarantee precisely for the knowledge that lives that way.

So, did melting the stack melt the editability? No, but it refracted it. The address book survives: classes are still rows you can point at, paste in, and delete. What changed is that the rows now come in two kinds, and the guarantees follow the kind, not the wish. Readout rows keep every proof from the list network: exact invariance, exact undo, zero gradient steps. Dynamics rows are more powerful, one paste re-carves the landscape at every depth at once, something a stack would charge twelve edits and 6,144 numbers for, and in exchange they demand a measurement culture: check the certificate, re-solve the fixed points, report the drift, temper the knob when the slope crosses one. An equation you can edit turns out to be an equation you must also audit. The Yat operator obliges on both counts, because the same kernel that makes its rows legible makes every one of these measurements a thing you can run, and this post just ran them.

@misc{bouhsine2026editafixedpoint,
  author       = {Bouhsine, Taha},
  title        = {Edit One Operator, Edit Every Depth},
  year         = {2026},
  month        = {jul},
  howpublished = {\url{https://tahabouhsine.com/blog/edit-a-fixed-point/}},
  note         = {Blog post, Records of the !mmortal Data Scientist}
}