Compression and compiler writing are just as important today as they were back then. And the closer to the real end of cheap hardware improvements we get the more important they will be. They are where real progress lies, the rest was just a temporary freebie.
Take cars: cars have a reasonable upper limit in physics so low that the difference between 'mild' (100 kph on the highway) and insane (300 kph+, no matter where) is a mere factor of three. And it took a good 100 years before that barrier was broken. It won't progress much beyond the standard of 100 or so kph under normal circumstances. There are just too many things that work against you when you go much faster than that including energy consumption, braking distance, the willingness of your vehicle to stay on the ground, attention requirements (ok, AI... someday...) and so on. So even if it is possible in exceptional cases it likely won't be the norm, even on closed over and evacuated freeways.
Aircraft did a little better, but even there the supersonic passenger jet came and went. Though it may be back one day.
In comparison computers have gotten many orders of magnitude faster in a much shorter time. But there is a physical limit, and we are very likely to run into that soon. And once that happens the only way to progress is to make more efficient use of the resources you've already got.
I don’t agree. Aircraft had their growth phase earlier, first flight in 1903 to landing on the moon in 1969. Airliners look like they did in the 60s because there’s nothing left to improve on with regards to efficiency. How many orders of magnitude are there between the wright flier and a 747?
We’re reaching the diminishing returns portion of the logistic curve for computing with regards to both things like hardware and compiling, compression, etc. It’s just hard to compare industries which are done growing with ones that still feel like they might be exponential forever.
It’s just hard to compare industries which are done growing with ones that still feel like they might be exponential forever.
Some industries never evolved too much. I don't see a big functional difference between cars a 100 years ago and cars now. And there were electric cars in 1900s.
A car is constraint by lots of things just like GP comment affirmed. Putting more gadgets in a car won't change what a car is or how it works at a fundamental level.
From the very first microprocessors to today's there is a big leap. If we apply that leap to cars they should be using nuclear fusion and take us to moon and back in seconds. And it's not like trillions of dollars weren't invested in car R&D in more than a century.
Software growth seems exponential only because of hardware and the Moore's Law. Which will come to an end soon.
Thanks to Moore's Law, we programmers, afforded to be careless, not think about performance or use unnecessary abstractions just out of esthetical or ideologic considerations. It allowed businesses to say that time to market trumps everything and customers should simply buy more powerful boxes or more boxes. Now this will come to an end soon and we will have to think about efficiency, speed and performance.
Or maybe chips will move from silicon to some alternative processes where growth is still possible.
You are making my point stronger rather than weaker: Aircraft have only become more efficient in the last 50 years or so and that's precisely the equivalent of better compilers and compression.
And the diminishing returns for computing are for the most part driven by the lack of investment in new ways of computing because we got so much for free. But once the free ride is over you can expect the research into efficiency to be picked up again because that will be the only field where real progress can be made. There have been a few half-assed (apologies to those involved, it was no doubt a ton of work) attempts at making computing fabrics and clockless machinery. And that's because by the time you have something working the scaling advantages have already overtaken your work and absolutely nobody is going to give up a working architecture for something unproven if it doesn't given an immediate return. Some pretty good ideas died like that.
Yes, for digital computing. But computing is on the cusp of becoming more non-determistic, which will allow analog computing to absolutely demolish digital computing in performance.
In comparison computers have gotten many orders of magnitude faster in a much shorter time. But there is a physical limit, and we are very likely to run into that soon. And once that happens the only way to progress is to make more efficient use of the resources you've already got.
Good points. And I will even argue that while we didn't hit a physical limit in hardware yet, it's still important to have a reasonable performance because not having performance means resources are wasted, money are wasted and it might lead to user frustration.
But it seems to not be the majority's opinion or at least not most of the business owners opinion.
Unfortunately until crappy and slow performing software starts losing business to well-implemented alternatives, it will never change. The current PM model is purely based on shipping fast, and any thought to performance is immediately dismissed as "pre-mature optimization." I've been consulting for companies that had to rolling reboot their entire cluster of app servers every couple of hours because a host of memory leaks (that also added O(n) runtime) would slowly exhaust resources. Rather than have someone fix the problem, they chose to invest in devops to automate the rolling restarts. This is not all that unusual in my experience, because customers will tolerate very sluggish response times so it's not worth fixing to the business.
Entirely my experience. I worked for a very large company and since our microservices were running in Kubernetes it was totally OK for the services to crash. We just logged the things in elastic stack, used data dog to monitor, used some SRE to restart services when things were messy.
I've lost days to solve a memory bug in one of the services and no one cared.
The original architect and programmers were gone, the engineering managers were pushing Clean, SOLID and design patterns hopping that will help and the domain was hard because we had to deal with real money while respecting tens of different laws and locales.
We tried at our best to do the impossible, we were understaffed by a factor of 5X.
And the services were crashing like crazy but it looked like a no downtime because we provisioned another POD in Kubernetes.
Provisioning was like: how many pods do we need? 3? Let's make it nine. How much memory do we need? Half a gig? Let's make it 4 gig to be sure.
So it was a mess and it worked and that mess continues to work somehow.
Not sure if mess driving engineering is a sound bussineses idea, though.
Erlang has taken 'let it crash' to entirely new heights, but that's from a top level viewpoint, components are allowed to crash but the service isn't. And you're still supposed to figure out why your service crashed but I'm pretty sure not everybody does that.
So as long as your supervision mechanism is bulletproof (supervision trees) then you can get away with this for very long.
Comments
Compression and compiler writing are just as important today as they were back then. And the closer to the real end of cheap hardware improvements we get the more important they will be. They are where real progress lies, the rest was just a temporary freebie.
Take cars: cars have a reasonable upper limit in physics so low that the difference between 'mild' (100 kph on the highway) and insane (300 kph+, no matter where) is a mere factor of three. And it took a good 100 years before that barrier was broken. It won't progress much beyond the standard of 100 or so kph under normal circumstances. There are just too many things that work against you when you go much faster than that including energy consumption, braking distance, the willingness of your vehicle to stay on the ground, attention requirements (ok, AI... someday...) and so on. So even if it is possible in exceptional cases it likely won't be the norm, even on closed over and evacuated freeways.
Aircraft did a little better, but even there the supersonic passenger jet came and went. Though it may be back one day.
In comparison computers have gotten many orders of magnitude faster in a much shorter time. But there is a physical limit, and we are very likely to run into that soon. And once that happens the only way to progress is to make more efficient use of the resources you've already got.
I don’t agree. Aircraft had their growth phase earlier, first flight in 1903 to landing on the moon in 1969. Airliners look like they did in the 60s because there’s nothing left to improve on with regards to efficiency. How many orders of magnitude are there between the wright flier and a 747?
We’re reaching the diminishing returns portion of the logistic curve for computing with regards to both things like hardware and compiling, compression, etc. It’s just hard to compare industries which are done growing with ones that still feel like they might be exponential forever.
Some industries never evolved too much. I don't see a big functional difference between cars a 100 years ago and cars now. And there were electric cars in 1900s.
A car is constraint by lots of things just like GP comment affirmed. Putting more gadgets in a car won't change what a car is or how it works at a fundamental level.
From the very first microprocessors to today's there is a big leap. If we apply that leap to cars they should be using nuclear fusion and take us to moon and back in seconds. And it's not like trillions of dollars weren't invested in car R&D in more than a century.
Software growth seems exponential only because of hardware and the Moore's Law. Which will come to an end soon.
Thanks to Moore's Law, we programmers, afforded to be careless, not think about performance or use unnecessary abstractions just out of esthetical or ideologic considerations. It allowed businesses to say that time to market trumps everything and customers should simply buy more powerful boxes or more boxes. Now this will come to an end soon and we will have to think about efficiency, speed and performance.
Or maybe chips will move from silicon to some alternative processes where growth is still possible.
You are making my point stronger rather than weaker: Aircraft have only become more efficient in the last 50 years or so and that's precisely the equivalent of better compilers and compression.
And the diminishing returns for computing are for the most part driven by the lack of investment in new ways of computing because we got so much for free. But once the free ride is over you can expect the research into efficiency to be picked up again because that will be the only field where real progress can be made. There have been a few half-assed (apologies to those involved, it was no doubt a ton of work) attempts at making computing fabrics and clockless machinery. And that's because by the time you have something working the scaling advantages have already overtaken your work and absolutely nobody is going to give up a working architecture for something unproven if it doesn't given an immediate return. Some pretty good ideas died like that.
But I suspect their day will come.
Yes, for digital computing. But computing is on the cusp of becoming more non-determistic, which will allow analog computing to absolutely demolish digital computing in performance.
Good points. And I will even argue that while we didn't hit a physical limit in hardware yet, it's still important to have a reasonable performance because not having performance means resources are wasted, money are wasted and it might lead to user frustration.
But it seems to not be the majority's opinion or at least not most of the business owners opinion.
Unfortunately until crappy and slow performing software starts losing business to well-implemented alternatives, it will never change. The current PM model is purely based on shipping fast, and any thought to performance is immediately dismissed as "pre-mature optimization." I've been consulting for companies that had to rolling reboot their entire cluster of app servers every couple of hours because a host of memory leaks (that also added O(n) runtime) would slowly exhaust resources. Rather than have someone fix the problem, they chose to invest in devops to automate the rolling restarts. This is not all that unusual in my experience, because customers will tolerate very sluggish response times so it's not worth fixing to the business.
Entirely my experience. I worked for a very large company and since our microservices were running in Kubernetes it was totally OK for the services to crash. We just logged the things in elastic stack, used data dog to monitor, used some SRE to restart services when things were messy.
I've lost days to solve a memory bug in one of the services and no one cared.
The original architect and programmers were gone, the engineering managers were pushing Clean, SOLID and design patterns hopping that will help and the domain was hard because we had to deal with real money while respecting tens of different laws and locales.
We tried at our best to do the impossible, we were understaffed by a factor of 5X.
And the services were crashing like crazy but it looked like a no downtime because we provisioned another POD in Kubernetes.
Provisioning was like: how many pods do we need? 3? Let's make it nine. How much memory do we need? Half a gig? Let's make it 4 gig to be sure.
So it was a mess and it worked and that mess continues to work somehow.
Not sure if mess driving engineering is a sound bussineses idea, though.
Erlang has taken 'let it crash' to entirely new heights, but that's from a top level viewpoint, components are allowed to crash but the service isn't. And you're still supposed to figure out why your service crashed but I'm pretty sure not everybody does that.
So as long as your supervision mechanism is bulletproof (supervision trees) then you can get away with this for very long.