What does that mean ? It doesn't make sense to me. The algorithm will work regardless of the magnitude of w relative to log(n).
Radix sort will be faster than n log(n) if w < log(n). I would add that it is only worth to use if w << log(n). In fact radix sort process the sorting by groups of bits, so w is usually smaller than the number of bits.
Other sorting techniques may be faster. For example, using radix sorting, you can sort n word-sized integers by scanning the data a few bits at a time. Each scan can be done in linear time, and the number of scans depends on the word length w. Hence, the time for radix sorting is proportional to nw. Here, it’s easy to make a mistake. On most computers, w is a small number (typically 32 or 64), and you might be tempted to state that radix sorting is a linear time sorting algorithm. Not so. The algorithm will only work if w ≥ log n. If not, you wouldn’t even be able to store the numbers. Since each memory address is a word consisting of w bits, the address space won’t accommodate n numbers if w < log n. Hence, in the unit-cost RAM model, radix sort also runs in time proportional to n log n.
He's essentially cheating. When he analyzes the algorithm presented, he says "I’ll now describe an algorithm that does not depend on the word length. For any word length w, w ≥ log n, it sorts n word-sized integers in time proportional to n log log n."
But that's entirely cheating. If you give the same assumption to radix sort, then you could say "For any word length w, w ≥ log n, radix sort sorts n word-sized integers in time proportional to n". Which is much better.
The only time radix sort actually goes slower than linear time is when the size of the integers is not polynomial in n. If the size of the integers is exponential n, I think this algorithm would similarly slow down, as it would take a linear number of words to represent a single integer, meaning that it would take log n reductions to get the integers to fit into a single word, not log log n as he assumes.
I think his argument goes as follows. If you have N numbers in memory, then you have to use pointers of size log N bits. Because pointers use the same number of bits as numbers, this means that the numbers have to be log N bits. Because radix sort does one O(N) pass per bit, this means that total time will be O(NlogN).
If we sort numbers of a fixed, known, size w', that is independent of machine word size w, then the argument breaks.
Comments
Isn't radix sort o(n) ?
What does that mean ? It doesn't make sense to me. The algorithm will work regardless of the magnitude of w relative to log(n).
Radix sort will be faster than n log(n) if w < log(n). I would add that it is only worth to use if w << log(n). In fact radix sort process the sorting by groups of bits, so w is usually smaller than the number of bits.
I will quote from the linked article:
Other sorting techniques may be faster. For example, using radix sorting, you can sort n word-sized integers by scanning the data a few bits at a time. Each scan can be done in linear time, and the number of scans depends on the word length w. Hence, the time for radix sorting is proportional to nw. Here, it’s easy to make a mistake. On most computers, w is a small number (typically 32 or 64), and you might be tempted to state that radix sorting is a linear time sorting algorithm. Not so. The algorithm will only work if w ≥ log n. If not, you wouldn’t even be able to store the numbers. Since each memory address is a word consisting of w bits, the address space won’t accommodate n numbers if w < log n. Hence, in the unit-cost RAM model, radix sort also runs in time proportional to n log n.
He's essentially cheating. When he analyzes the algorithm presented, he says "I’ll now describe an algorithm that does not depend on the word length. For any word length w, w ≥ log n, it sorts n word-sized integers in time proportional to n log log n."
But that's entirely cheating. If you give the same assumption to radix sort, then you could say "For any word length w, w ≥ log n, radix sort sorts n word-sized integers in time proportional to n". Which is much better.
The only time radix sort actually goes slower than linear time is when the size of the integers is not polynomial in n. If the size of the integers is exponential n, I think this algorithm would similarly slow down, as it would take a linear number of words to represent a single integer, meaning that it would take log n reductions to get the integers to fit into a single word, not log log n as he assumes.
I think his argument goes as follows. If you have N numbers in memory, then you have to use pointers of size log N bits. Because pointers use the same number of bits as numbers, this means that the numbers have to be log N bits. Because radix sort does one O(N) pass per bit, this means that total time will be O(NlogN).
If we sort numbers of a fixed, known, size w', that is independent of machine word size w, then the argument breaks.