Most code only needs to work

Getting a model to produce output that runs is one thing and getting output that’s good and maintainable is supposed to be a different and much harder thing.

Except almost nobody is prompting for good, maintainable code in the first place. There’s a collective assumption that this is what we’re all aiming at and from there people assume it must be what the model is aiming at too. That isn’t true and there’s no reason it would be. A model doesn’t care about our goals. It’s solving a problem probabilistically from its training data, the context you gave it and the way you asked.

Think about what that implies for the code you get back. Take all the code in the world and put it through a mixer and what comes out isn’t the Platonic ideal of the best possible build. It’s closer to a Lovecraftian horror that happens to produce the right output and the fact that it produces the right output is really remarkable rather than a shame.

That’s a positive finding, oddly. It means that with better prompting and context and training we might be able to guide these things toward knowing what good and bad look like because we know what good and bad look like and that knowledge is expressible. The gap isn’t a wall. It’s an alignment problem in the ordinary sense and those get worked on.

The more uncomfortable half is that good, maintainable code is often not what a business wants from a developer either.

In companies everywhere the actual goal is whatever works right now and maintainability can look after itself. When a business is three months from running out of money, time spent writing something you could still be extending in ten years feels like exactly what it is which is effort spent on a future that may not arrive. That’s not a failure of engineering culture. It’s a rational response to the situation.

Which means that for most code that gets written, it doesn’t need to be good and it doesn’t need to be maintainable. It needs to work. And if you go and look at what businesses are actually running, an enormous amount of it doesn’t even manage that.

That’s the honest frame for what you’re actually using these tools for. They’re a step forward at getting things to work in the first place which is the bar most software is failing at rather than the bar we talk about in conference talks. Judging them against our stated standard misses what they’re being used for.

If we can get to something like bug free at the level of individual units, that’s really useful. Above that level, at logic and architecture and security, it still has to come from a person because none of these things can hold the context of an entire application yet. Tech debt lives almost entirely at that higher level and I don’t think we get there for a long time.

The uncomfortable follow-on is what this does to how you should judge your own work. If you’ve spent a career caring about whether code is good and the market is mostly buying whether code works, then the thing you’ve been optimising and the thing being paid for have never quite been the same. That gap was always there. It just didn’t matter much while producing working code was slow enough that quality and speed travelled together.

Which is why the arguments about whether generated code is good enough keep going past each other. One side is measuring against what a careful engineer would write and the other is measuring against what actually shipped last quarter, and those two standards are much further apart than anybody involved wants to say out loud.