When a Config Change Is Really an Acquisition

My CMDB console runs a small local model, and its answers were not as good as I wanted. A popular fix arrived on schedule: put a model router in front of it. One package, one line of configuration, and the console can reach dozens of hosted models, picking the best one per question. Better answers, more resilience, done by lunch.

I wrote it up as a decision instead. That turned out to be the only interesting part.

What the one line actually does

Every question to the console carries context: retrieved architecture notes, record names, sometimes live values from the CMDB. Today that context never leaves the machine. With a router configured, it is sent to whichever provider the router picks, under whatever terms that provider has.

That is not a configuration change. It is a new external data processor. In most enterprises that means a security review, a third-party risk review and a contract — the same path as buying any other service. The router skips all three by being easy to install.

I don't fear complex decisions. I fear the ones that fit in a config file.

Why it gets made by convenience

Acquisitions are visible. They have a requester, a budget line and a review. Configuration changes have none of those. They get made at 4 p.m. by someone fixing a symptom, and they are trivially reversible — except for the data. You can remove the router tomorrow. You cannot recall what it already sent.

Test the premise first

The case for the router rested on a hypothesis: the model is too small. So I tested that before arguing about anything else. The evaluation said the bottleneck was retrieval shape — chunks that held summaries only, and text truncated before the answer appeared. A bigger model reading the same bad context gives a more fluent wrong answer. Fixing retrieval moved the scores. Swapping models would have moved the bill.

The questions that turned it into a decision

• Data egress. What leaves the boundary, to whom, under which terms?
• Supply chain. What does the package pull in, and who maintains it?
• Governance path. Does this trigger security, third-party and contract review? If yes, it is an acquisition, whatever the install command says.
• Quality gain. Is the improvement measured, or assumed?
• Durability. Does it survive the provider changing models, prices or terms?
• Transparency. Will the user know which model answered?

What I chose instead

A fallback chain on the same local endpoint, with the answering model disclosed on every response. No new data processor, no new review, and if the primary model fails the user still gets an answer — with a label saying which model gave it. Less exciting. Considerably fewer phone calls from security.

If a hosted model is ever genuinely needed, it goes through procurement like anything else. Being able to do it is not permission to do it.

Copy this: the config-or-acquisition test

## Is this a config change or an acquisition?
Treat it as an ACQUISITION (security + third-party + contract review) if ANY are true:
- Data leaves our boundary to a party we have no contract with.
- A new external processor sees prompts, records or credentials.
- The provider can change models, terms or retention without telling us.
- It cannot be fully undone (data already sent cannot be recalled).
Otherwise it is a config change - still record: what changed, why,
the measured gain, and how to roll it back.
Before either: test the premise. Measure that the problem is the one
the change claims to fix.