Does the LLM value training over ergonomics

I’m pretty sure I’m not the first person to have said this, I’m not even going to bother searching it to see if someone has - but today I found myself deep in a thought experiment.

It went something like this:

If you accept that one-day most software development will be done with LLM assistance then the best and most consistent results wil come from using tools where the LLM has a higher chance of success.

I considered this yesterday because I quite like Rust and I was considering whether to use it or not in my solution. To me its code is quite readable, it executes pretty fast and the memory errors you get from writing code with it are limited.

But I’m not writing the code anymore, the LLM is, and if the training data for using Rust is a lot smaller than the training data for other systems programming languages like C (or C++) then the Rust guarantees don’t really mean much anymore because the LLM can write clean and safe C code, even if I can’t.

So that leads me to this:

With a sufficiently small training set the advantages of an ergonomic tool might be outweighed by a less ergonomic tool with a larger training set.

An example of this might be C++, that added objects to C. Adding objects gave C++ a useful way of thinking about software design and abstraction that is harder to do in C. Yes you can do objects in C, but there is cognitive load needed to do objects in pure C which is a bit distracting and diminishes the benefits.

But the LLM doesn’t need quite the same mental aids that we do to make development easier. I asked Claude to tell me what languages it was most happy with, it replied: “C and Golang”. When pressed it revealed the reason for that was the wide training data set and the good tools available for mitigating any particular short-comings of those languages that might exist. Assembler was something it didn’t care for because of the portability issues.

I then asked what programming techniques Claude considers most challenging and it answered distributed systems and concurrency. I’m not sure if this is an echo of what people say in the training data, or whether those things are actually harder for Claude. But empirically I’ve also noticed that Claude steers away from using threading. That said I’m sure if I constructed a problem that could only be solved by threads then it would give me a recommended solution involving threads. But … there is a whole school of programming that tries to avoid using threads at all costs because you can often do the job just as well without them, it’s just a different style of software. So say that it’s true that LLMs don’t like threads and don’t recommend solutions involving them, won’t it mean that fewer programs of the future will use them?

That led me to this:

If someone discovers that there is a technology that is better-suited for LLMs to use for some class of problem, then it won’t matter how old it is or how awkward it is for humans to use. Since new greenfield projects could well do as I’m doing and drift towards using the most natural fit for the LLM.

Now I agree that as the human in charge of the new greenfield project I need to be able to understand the code the LLM produces but C is easy enough to follow when read aloud, what’s less hard to follow is the memory management part of what the author was doing. But here’s the thing so long as it works within my supplied constraints does any of that matter?

So for the LLM there is now a trade-off between being more familiar with C but needing to do more leak testing versus; being less familiar with Rust, having to fight the borrow-checker, but not needing to do the leak testing. When I hear that I’m less inclined to choose Rust, C sounds safer and easier.

Which brings me to the last point:

Things that LLMs struggle with due to lack of training data, translate into higher token costs in rework. Higher token costs could lead to those technologies being abandoned eventually. Even though they are net good technological advances in themselves, the ergonomics of those solutions matters less. Since the ergonomics layer has now shifted to the chat window.

Note that this is still a thought experiment, I can’t back any of these claims up.

But I find that fascinating, because taken to the logical extreme it becomes very hard for new technologies of the future to change what we do and how we work. Whereas before a programming language may be created by you might writing a few university papers, doing some conferences, getting the students using it and then hoping they take it with them into industry. Now you have to get everyone’s LLM to be trained on it first. And that sounds incredibly hard.

I don’t quite know what to make of it. If it comes to pass in some ways it’s good, it means that we don’t waste energy on tools, we concetrate energy on solutions that bring benefits. And the business analyst in me definitely loves this. The economist in me thinks of the increase in productivity we will gain from having a simpler technology stack when all these tools disappear will be fantastic.

Now … the philospher in me wonders if we will give up something fundamental about what it means to solve problems, which is in turn what makes us human. Something I know others have commented on.

Whilst … the creative/artist in me just weeps for the joy of making a thing that people want to use.