We're [low-level programmers] not dead, we're pining for the fjords. Low level programming is a dark art, and there are naturally plenty of people celebrating how fantastic it is that they don't _have_ to know this stuff. Don't celebrate your ignorance; you may be 2-4 orders of magnitude off after your performance is sucked away by all those layers of abstraction.
My take on it is that "low-level stuff" isn't going away any time soon, and that there will always be a equilibrium reached between those who understand it and are fluent with it (and, sometimes, are going to be able to write code that is 5-100x faster) and those who don't.
It's orthogonal to an understanding of algorithms. No amount of bit-bashing can fix really poor choices of algorithms (N^2 vs NlogN, say, on a big input). That being said, there are a lot of tasks where everyone is going to land on the same linear or logN basic algorithm and the main difference is going to be the algorithm that misses cache frequently and is stuffed with branch mispredicts and pipeline stalls vs. some algorithm that avoids all these problems and runs 10x faster. Sometimes you've just got to go do something to every bit of data and there's no classic algorithmic trick.
For example, I helped a former academic colleague write the 'worlds fastest floating point minimum' routine using SSE and lots of unrolling/software pipelining and I think he got about 8-10x (he had a library for most of the common operations like vector add, mul, etc but it didn't do 'min'). No amount of rampant algorithmic cleverness would avoid the need to look at each data element once when trying to calculate the min of a vector and if you're doing WORSE than linear, you've got real problems.
The major point that I've discovered from doing this stuff for years (on considerably more complex cases than the example above) is that programming efficiently for a modern architecture is qualitatively different than designing algorithms for the abstract '1 operation counts as 1 operation' machine in your average algorithms textbook. Oddly, quite a bit of improvement in my scalar, non-parallel programming came about after having used CUDA fairly intensively - if you're writing code for a Core 2 Duo or beyond, you're already parallel programming even if you're designing code that's single-threaded. Understanding how to rethink your algorithm to have data-parallelism (not to mention using SSE) is just plain conceptually different and more akin to parallel programming than scalar programming.
Knowing when to do this is important; I like bashing out a quick Python script as much as the next guy, or perhaps some pretty random C++ STL code that's probably an order of magnitude from where it should be (not because the STL is bad but because, say, I've been bone-lazy with design). So all those 'I have 10 layers of abstraction above this level" nincompoops shouldn't get too smug; we (low-level guys) can go there too - just because we know how to optimize C/asm loops to within an inch of their lives doesn't mean that we're going to compulsively do it with every last one.
Also worthy of note - the right thing to do changes frequently; some of the resources (especially on branch prediction) listed are already out of date. Don't bring P4 knowledge to a Sandy Bridge fight. Some bit twiddling hacks are great, others (especially ones that assume multiply is ultra-expensive) are obsolete on recent x86. The concepts still keep their validity a lot more than, say, all that newfangled crud that you youngsters fill your heads with (LAMP stacks, etc.) :-)
No amount of bit-bashing can fix really poor choices of algorithms (N^2 vs NlogN, say, on a big input).
Its also worth noting that sometimes an algorithm with higher O() complexity could actually perform better. Eg, a brute-force linear search may be faster than a binary search if the elements can be efficiently cache prefetched, or if the entire dataset fits into cache.
The major point that I've discovered from doing this stuff for years (on considerably more complex cases than the example above) is that programming efficiently for a modern architecture is qualitatively different ...
I think this is a great point. Low-level programming has always required an in-depth understanding of the underlying hardware and that hardware is changing as fast as ever today. It may be true that a large percentage of development can live happily in a land of high-level abstractions, but there will always be a need for low-level optimization and that space is still rapidly changing in very interesting ways.
I'm having an extremely hard time finding a remote dev job, as I have years of C/C++ knowledge and industry experience. If I worked in webdev, I probably wouldn't say the same.
RAD's only ten programmers and that number stays more or less constant. The way everyone is hired is unique and inevitably somewhat odd. But it usually involves knowing Jeff or Jeff knowing of you for a long time and eventually he makes an offer when it makes sense.
Comments
We're [low-level programmers] not dead, we're pining for the fjords. Low level programming is a dark art, and there are naturally plenty of people celebrating how fantastic it is that they don't _have_ to know this stuff. Don't celebrate your ignorance; you may be 2-4 orders of magnitude off after your performance is sucked away by all those layers of abstraction.
My take on it is that "low-level stuff" isn't going away any time soon, and that there will always be a equilibrium reached between those who understand it and are fluent with it (and, sometimes, are going to be able to write code that is 5-100x faster) and those who don't.
It's orthogonal to an understanding of algorithms. No amount of bit-bashing can fix really poor choices of algorithms (N^2 vs NlogN, say, on a big input). That being said, there are a lot of tasks where everyone is going to land on the same linear or logN basic algorithm and the main difference is going to be the algorithm that misses cache frequently and is stuffed with branch mispredicts and pipeline stalls vs. some algorithm that avoids all these problems and runs 10x faster. Sometimes you've just got to go do something to every bit of data and there's no classic algorithmic trick.
For example, I helped a former academic colleague write the 'worlds fastest floating point minimum' routine using SSE and lots of unrolling/software pipelining and I think he got about 8-10x (he had a library for most of the common operations like vector add, mul, etc but it didn't do 'min'). No amount of rampant algorithmic cleverness would avoid the need to look at each data element once when trying to calculate the min of a vector and if you're doing WORSE than linear, you've got real problems.
The major point that I've discovered from doing this stuff for years (on considerably more complex cases than the example above) is that programming efficiently for a modern architecture is qualitatively different than designing algorithms for the abstract '1 operation counts as 1 operation' machine in your average algorithms textbook. Oddly, quite a bit of improvement in my scalar, non-parallel programming came about after having used CUDA fairly intensively - if you're writing code for a Core 2 Duo or beyond, you're already parallel programming even if you're designing code that's single-threaded. Understanding how to rethink your algorithm to have data-parallelism (not to mention using SSE) is just plain conceptually different and more akin to parallel programming than scalar programming.
Knowing when to do this is important; I like bashing out a quick Python script as much as the next guy, or perhaps some pretty random C++ STL code that's probably an order of magnitude from where it should be (not because the STL is bad but because, say, I've been bone-lazy with design). So all those 'I have 10 layers of abstraction above this level" nincompoops shouldn't get too smug; we (low-level guys) can go there too - just because we know how to optimize C/asm loops to within an inch of their lives doesn't mean that we're going to compulsively do it with every last one.
Also worthy of note - the right thing to do changes frequently; some of the resources (especially on branch prediction) listed are already out of date. Don't bring P4 knowledge to a Sandy Bridge fight. Some bit twiddling hacks are great, others (especially ones that assume multiply is ultra-expensive) are obsolete on recent x86. The concepts still keep their validity a lot more than, say, all that newfangled crud that you youngsters fill your heads with (LAMP stacks, etc.) :-)
No amount of bit-bashing can fix really poor choices of algorithms (N^2 vs NlogN, say, on a big input).
Its also worth noting that sometimes an algorithm with higher O() complexity could actually perform better. Eg, a brute-force linear search may be faster than a binary search if the elements can be efficiently cache prefetched, or if the entire dataset fits into cache.
The major point that I've discovered from doing this stuff for years (on considerably more complex cases than the example above) is that programming efficiently for a modern architecture is qualitatively different ...
I think this is a great point. Low-level programming has always required an in-depth understanding of the underlying hardware and that hardware is changing as fast as ever today. It may be true that a large percentage of development can live happily in a land of high-level abstractions, but there will always be a need for low-level optimization and that space is still rapidly changing in very interesting ways.
I'm having an extremely hard time finding a remote dev job, as I have years of C/C++ knowledge and industry experience. If I worked in webdev, I probably wouldn't say the same.
That's probably as much to do with different parts of the industry's expectations/requirements when it comes to on-site or off-site work.
Is RAD looking for an experienced graphics programmer / etc, by chance?
Sorry, RAD doesn't really hire in the traditional sense.
If you're looking for game development work, I can see how being stuck in St. Louis must be tough. Good luck!
Thanks!
Out of curiosity, what's different about RAD's hiring?
RAD's only ten programmers and that number stays more or less constant. The way everyone is hired is unique and inevitably somewhat odd. But it usually involves knowing Jeff or Jeff knowing of you for a long time and eventually he makes an offer when it makes sense.
That's so cool!