Model risk management was built for a different machine

Model Risk Management — the framework most institutions still run on — was written for machines that sat still. You specified the model, validated it against historical data, documented the assumptions, and re-validated each cycle. That discipline was hard won and it worked, because the thing being governed had three properties that no longer hold.

It was yours. Tight controls, known logic. It changed only when you changed it. Today the underlying models are a vendor’s, and they move underneath you. A version rolls forward, weights shift, a safety layer is re-tuned, and the process you validated in March is not the process answering in July. Nobody sent you a change ticket.

Its inputs were bounded. You knew the distribution, so you could measure whether you were inside it. That input space is now natural language, which is to say unbounded, and the control distinctions and datasets you have relied on historically quietly stop carrying weight.

Its output was a number in a range. Now the output is a judgment, or an action, moving through a workflow that does not pause for review.

So periodic validation of a static artifact is the wrong shape of control. Not too weak — aimed at the wrong object.

What transfers is the instinct: govern the thing that produces the number. What changes is that the thing is no longer the model. It is the inference — the AI’s changing answer to the same question — embedded in your control set. That means:

• Pin versions, and detect when they move.

• Constrain what the system can do unassisted, and make the boundary explicit rather than cultural — enforced human gates.

• Capture evidence per call rather than per quarter.

• Monitor the decisions that resulted, not the metrics of the model that suggested them.

The framework will get rewritten eventually. Controllers are going to be asked to sign something well before that happens.