Honestly, having watched people argue about what simple is for about the last 10 years, I've pretty much settled on it not being a well-defined term. We know complex when we see it for sure, at least when it is present in quantity, but simplicity is not just the absense of complexity. There's at least three concepts we're all trying to stuff into the same word, and they are not only not "orthogonal" they are often in conflict with each other. I don't even think it can be rehabilitated, it can only really be abandoned, to clear the way to trying to characterize the multiple concepts we're trying to stuff into this one word.
It is especially dangerous when something is "good" and people try to appropriate the term to appropriate the goodness of the term, as if goodness flows from a term to the thing it is attached to rather than the other way around. "Simple" is good so my good thing must be "simple" to be "good". But it doesn't. Simple can even be bad, in the wrong place or in the wrong sort of "simple" for a given job.
Reducing entropy or cross entropy fits intuition and theory and it's what LLMs target during training in order to model languages, to model the world and to enable calibrated reasoning across uncertainty and gray areas. I think you can use entropy reduction as a good definition of simplicity even though it's not always "simple" to understand what that entails.
"Few tokens" - perhaps the most literal simplicity, literally, it doesn't use many tokens to do the job. But as the article points out, that doesn't necessarily fit with...
"Easy to reuse" - This is that simplicity that functional programming aspires to, where you craft some precise abstraction that somehow captures something like "monad". Haskell is full of this sort of simplicity, oozing out of every pore, but people generally think of it as a very complex and hard langauge, contrasting...
"Easy to understand" - As in, not cognitively complex. It is amazing how quickly things that I would otherwise describe as very simple still blow out our little minds. Consider the first time you saw quicksort... or even how it feels now. It's not a lot of tokens, but it's twisty and recursive and especially if you're not mathematically trained and in practice it's easy to call it more "complicated" than a CRUD form that takes in and validates 10 parameters in a straightforward way, even though in terms of what is actually happening the CRUD form may be doing vastly more than the little quicksort algorithm. It just isn't being twisty, recursive, and subtle in how it does it.
And I'm just filling out the first three that come to mind. Note these are not always in conflict by any means... but they certainly aren't always in harmony with each other either.
(One might argue "easy to reuse" is more about the complexity of the code doing the reusing, but I feel like this is definitely something people mean when they talk about the simplicity of code.)
Comments
Honestly, having watched people argue about what simple is for about the last 10 years, I've pretty much settled on it not being a well-defined term. We know complex when we see it for sure, at least when it is present in quantity, but simplicity is not just the absense of complexity. There's at least three concepts we're all trying to stuff into the same word, and they are not only not "orthogonal" they are often in conflict with each other. I don't even think it can be rehabilitated, it can only really be abandoned, to clear the way to trying to characterize the multiple concepts we're trying to stuff into this one word.
It is especially dangerous when something is "good" and people try to appropriate the term to appropriate the goodness of the term, as if goodness flows from a term to the thing it is attached to rather than the other way around. "Simple" is good so my good thing must be "simple" to be "good". But it doesn't. Simple can even be bad, in the wrong place or in the wrong sort of "simple" for a given job.
Because familiarity is a confound for intuition about complexity, even this is not always true.
Maxwell's equations will look complex to the uninitiated, and can represent the pinnacle of simplicity to those who already understand them.
Reducing entropy or cross entropy fits intuition and theory and it's what LLMs target during training in order to model languages, to model the world and to enable calibrated reasoning across uncertainty and gray areas. I think you can use entropy reduction as a good definition of simplicity even though it's not always "simple" to understand what that entails.
https://benoitessiambre.com/entropy.html
Could you try to define these three separate concept?
I see at least:
"Few tokens" - perhaps the most literal simplicity, literally, it doesn't use many tokens to do the job. But as the article points out, that doesn't necessarily fit with...
"Easy to reuse" - This is that simplicity that functional programming aspires to, where you craft some precise abstraction that somehow captures something like "monad". Haskell is full of this sort of simplicity, oozing out of every pore, but people generally think of it as a very complex and hard langauge, contrasting...
"Easy to understand" - As in, not cognitively complex. It is amazing how quickly things that I would otherwise describe as very simple still blow out our little minds. Consider the first time you saw quicksort... or even how it feels now. It's not a lot of tokens, but it's twisty and recursive and especially if you're not mathematically trained and in practice it's easy to call it more "complicated" than a CRUD form that takes in and validates 10 parameters in a straightforward way, even though in terms of what is actually happening the CRUD form may be doing vastly more than the little quicksort algorithm. It just isn't being twisty, recursive, and subtle in how it does it.
And I'm just filling out the first three that come to mind. Note these are not always in conflict by any means... but they certainly aren't always in harmony with each other either.
(One might argue "easy to reuse" is more about the complexity of the code doing the reusing, but I feel like this is definitely something people mean when they talk about the simplicity of code.)