Guys! The point of this article is not to prescribe the only method of displaying human-readable file sizes. Obviously one could use `ls -lh`; the author clearly demonstrates that he is willing and able to read man pages to find answers.
Rather, this is a pretty interesting look into what it actually entails to make what ought to be a very simple and straightforward change.
It turns out that these simple changes are hard! Not just in identifying the piece of code to modify, but that man pages are often incomplete or unclear. It also illustrates the complexities behind making software portable - in this case, using the nation-neutral place separator. It also reminds us that solving what is on the surface a simple problem lets one uncover all sorts of interesting and messy details underneath - including more problems to solve!
These are steps that he'd have to take no matter what the code or feature. This article is not "complexity for complexity's sake", it's illustrating the complexity of making changes to any piece of code - and that it is surprisingly difficult for something that one would think is very easy!
On the other hand, you could consider it a cautionary tale about not reinventing wheels, because a problem that may seem trivial at first often turns out to be far more complex than expected.
This is why you try to re-use work when possible, rather than endlessly reinventing things, because while sure, adding a comma to the printf string is easy enough, your assumptions (English locale, compiler not trying to be clever) are going to quickly become visible as things fall apart because your assumptions aren't in line with the system's assumptions.
What this story really demonstrates is that without a clear understanding of how a system is designed and the basic assumptions it makes, just "hacking on the code" is just as likely to break things as it is to fix them.
You assume that systems tend to be well-designed and the basic assumptions it makes are justifiable, and that the systems faithfully implement that design and those assumptions.
It sounds like the author found a bunch of bugs in the process of making a simple code change. That happens pretty frequently, and doesn't mean that the author should let the priests of the cathedral deal with this UNIX thing that is too complicated for the laity to hack on. It just means there is no priesthood.
It is not very easy because it is an unusual request.
But I still wonder if this is easier than ls -l | sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta' ??
The investigations would be interesting it they were more complete, i.e. if the actual result was a change in the locale which could be appliable to other tools printing numbers besides ls (in the author TODO).
I mean, will it work with bc?
At the moment it's not better than an shell script alias giving the output to sed, but it is more complex - you have to recompile a binary for every OS you use.
It's a 10 seconds trick that did the job and did not require recompiling.
EDIT: seeing how it has been downvoted, IMHO hacking is all about time and effectiveness. if you believe fixing ls print formats is such a crucial problem that it requires more than seconds of your time, we have different values.
Feel free to support the argument by showing your skills and improving the example.
You are entirely missing the point of this whole thing.
It is exactly about showing how hard it is to get a trivial change right in every detail. Your 10 second hack is what is wrong with 10 second hacks in general and even with most real solutions that are not carefully thought out.
It's not only that the devil is in the details it is all details. And you need to get all of them right, not just the current subset of the problem that you happen to be working on.
While I think that is the point, I don't know if was the author's originally intended point. You see 'hacking it' would imply just getting it to work, I expected to see something, well hackish, like replace the format specifier with %s and then wrap the number in a function call to something which read an environment variable to figure out how to print the number. That would be a hack, what Greg is doing is engineering a change to ls(1) which allows for pretty printing the numbers. So to my way of thinking the title is wrong, it sets the expectation of a hack and leads to an engineering exercise.
I think that's reflective of how experience levels change your perspective on how to approach a problem.
Someone with command line experience that is mostly a user of system utilities would think passing the output of ls through a filter is the way to go whereas someone with C experience that already has insight in how Unix is architected would do something more along the lines of what the author is doing. That's pretty much how unix got built to begin with, and probably both parties would qualify their solution as a 'hack'.
"Hacker" has many meanings. A quick read of the Wikipedia entry for the term shows you that. RFC 1392 defines a hacker as "A person who delights in having an intimate understanding of the internal workings of a system, computers and computer networks in particular." The only way to tell which definition is in use is via context, and for this case the meaning is unambiguously aligned with the RFC 1392 definition.
(Note that this RFC comes from the days when 'hacker' was becoming a widespread term for someone who breaks into computer systems; the RFC attempts to distinguish between a 'hacker' and a 'cracker.')
It's a disgusting hack and it works very well. I didn't find anything useful in the man page so I wrote this instead. Looks like there is something in the man page but I think my hack was probably faster.
Your solution is the better one, but that does not take anything away from memset's argument, though... This is not about the solution. This is about the problem of having all these yaks to shave just to add a comma to some program's output.
> It turns out that these simple changes are hard! Not just in identifying the piece of code to modify, but that man pages are often incomplete or unclear.
This is a great shame. I like OpenBSD's approach to man pages - incorrect documentation is a bug and can be as severe as a bug in code; correct documentation is important.
Fixing up man pages is something that non-technical volunteers could help with, except when it's hard to grok what the code actually does vs what it should do.
> Fixing up man pages is something that non-technical volunteers could help with
Another problem with that is that most tools used in this process are made with technical users in mind. A lot of people can expand documentation, but sending manpages patches in a bug tracker is a technical step.
Comments
Guys! The point of this article is not to prescribe the only method of displaying human-readable file sizes. Obviously one could use `ls -lh`; the author clearly demonstrates that he is willing and able to read man pages to find answers.
Rather, this is a pretty interesting look into what it actually entails to make what ought to be a very simple and straightforward change.
It turns out that these simple changes are hard! Not just in identifying the piece of code to modify, but that man pages are often incomplete or unclear. It also illustrates the complexities behind making software portable - in this case, using the nation-neutral place separator. It also reminds us that solving what is on the surface a simple problem lets one uncover all sorts of interesting and messy details underneath - including more problems to solve!
These are steps that he'd have to take no matter what the code or feature. This article is not "complexity for complexity's sake", it's illustrating the complexity of making changes to any piece of code - and that it is surprisingly difficult for something that one would think is very easy!
On the other hand, you could consider it a cautionary tale about not reinventing wheels, because a problem that may seem trivial at first often turns out to be far more complex than expected.
This is why you try to re-use work when possible, rather than endlessly reinventing things, because while sure, adding a comma to the printf string is easy enough, your assumptions (English locale, compiler not trying to be clever) are going to quickly become visible as things fall apart because your assumptions aren't in line with the system's assumptions.
What this story really demonstrates is that without a clear understanding of how a system is designed and the basic assumptions it makes, just "hacking on the code" is just as likely to break things as it is to fix them.
You assume that systems tend to be well-designed and the basic assumptions it makes are justifiable, and that the systems faithfully implement that design and those assumptions.
It sounds like the author found a bunch of bugs in the process of making a simple code change. That happens pretty frequently, and doesn't mean that the author should let the priests of the cathedral deal with this UNIX thing that is too complicated for the laity to hack on. It just means there is no priesthood.
If your definition of "breaking" things is "they don't work on the first try", everything I've ever tried to do was broken.
It is not very easy because it is an unusual request.
But I still wonder if this is easier than ls -l | sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta' ??
The investigations would be interesting it they were more complete, i.e. if the actual result was a change in the locale which could be appliable to other tools printing numbers besides ls (in the author TODO).
I mean, will it work with bc?
At the moment it's not better than an shell script alias giving the output to sed, but it is more complex - you have to recompile a binary for every OS you use.
When I use your sed transformation on one of my directories, I see entries like:
I don't want to see the year written "2,012", and the file name is 'pubchem_10000" not "pubchem_10,000".It's a 10 seconds trick that did the job and did not require recompiling.
EDIT: seeing how it has been downvoted, IMHO hacking is all about time and effectiveness. if you believe fixing ls print formats is such a crucial problem that it requires more than seconds of your time, we have different values.
Feel free to support the argument by showing your skills and improving the example.
You are entirely missing the point of this whole thing.
It is exactly about showing how hard it is to get a trivial change right in every detail. Your 10 second hack is what is wrong with 10 second hacks in general and even with most real solutions that are not carefully thought out.
It's not only that the devil is in the details it is all details. And you need to get all of them right, not just the current subset of the problem that you happen to be working on.
While I think that is the point, I don't know if was the author's originally intended point. You see 'hacking it' would imply just getting it to work, I expected to see something, well hackish, like replace the format specifier with %s and then wrap the number in a function call to something which read an environment variable to figure out how to print the number. That would be a hack, what Greg is doing is engineering a change to ls(1) which allows for pretty printing the numbers. So to my way of thinking the title is wrong, it sets the expectation of a hack and leads to an engineering exercise.
I think that's reflective of how experience levels change your perspective on how to approach a problem.
Someone with command line experience that is mostly a user of system utilities would think passing the output of ls through a filter is the way to go whereas someone with C experience that already has insight in how Unix is architected would do something more along the lines of what the author is doing. That's pretty much how unix got built to begin with, and probably both parties would qualify their solution as a 'hack'.
It's all perspective.
"Hacker" has many meanings. A quick read of the Wikipedia entry for the term shows you that. RFC 1392 defines a hacker as "A person who delights in having an intimate understanding of the internal workings of a system, computers and computer networks in particular." The only way to tell which definition is in use is via context, and for this case the meaning is unambiguously aligned with the RFC 1392 definition.
(Note that this RFC comes from the days when 'hacker' was becoming a widespread term for someone who breaks into computer systems; the RFC attempts to distinguish between a 'hacker' and a 'cracker.')
The easiest and naive way to fix that is to buffer the whole thing. Don't use this to do `ls-comma -lR ~`, but only on small directory hierarchies.
https://github.com/samsonjs/bin/blob/master/ls-comma
It's a disgusting hack and it works very well. I didn't find anything useful in the man page so I wrote this instead. Looks like there is something in the man page but I think my hack was probably faster.
this should fix that. ls -l | sed -e :a -e 's/\s\(.*[0-9]\)\([0-9]\{3\}\) /\1,\2 /;ta'
correction: sorry it will only fix the modifications to the filename - the year is still broken.
The following modifies only the file size:
Yes. So long as you only want to modify ls -l. And don't care about column alignment. The original author wants output to look like:
With your perl one-liner on my directory I get mis-aligned columns: That's ugly. It should be:Your solution is the better one, but that does not take anything away from memset's argument, though... This is not about the solution. This is about the problem of having all these yaks to shave just to add a comma to some program's output.
> It turns out that these simple changes are hard! Not just in identifying the piece of code to modify, but that man pages are often incomplete or unclear.
This is a great shame. I like OpenBSD's approach to man pages - incorrect documentation is a bug and can be as severe as a bug in code; correct documentation is important.
Fixing up man pages is something that non-technical volunteers could help with, except when it's hard to grok what the code actually does vs what it should do.
> Fixing up man pages is something that non-technical volunteers could help with
Another problem with that is that most tools used in this process are made with technical users in mind. A lot of people can expand documentation, but sending manpages patches in a bug tracker is a technical step.
If one was so non-technical as to be incapable of using diff and a bug tracker, perhaps they shouldn't be writing manpages.
Sure, they probably shouldn't be creating man pages. But they could be great at copy editing and polishing existing man pages.
Or they could be expert translators.
So it's a shame that man pages are not as good as they could be.
There are other forms of documentation, but it'd be nice if man pages were the best the could possibly be.