Python's hash table type, dict, promises that it remembers insertion order. If you put the mapping 5 => "dog" in first, then when we ask what is in the dict we'd get told 5 => "dog" first, duh.
Historically Python had a more conventional but very, very badly implemented hash table type, the "I can't believe it can sort"† of hash tables. Somebody wanted a hash table type which remembers insertion order because Python programmers have a bad habit of writing "Golden tests" in which that order matters even though in a good modern hash table type it's not guaranteed, so they built one. But because the built-in hash table type was garbage, this new OrderedDict type was much faster and much smaller despite solving a more difficult problem.
For a little while it was unclear if Python would decide to rewrite their dict type to have decent performance or just embrace this new better alternative and then they decided that because it's beginner friendly they will just embrace the OrderedDict and require that this type has ordering.
However, a good hash table doesn't inherently have this property, and that goes for the Swiss Table the same as other common designs. So they can't swap dict out for a Swiss Table without breaking their own promise that the dict type preserves insertion order.
If you're used to a language where this doesn't happen such as Rust, or C++ or Java or any number of other programming languages, that insertion ordering rule seems crazy, but if you've never used a programming language at all before and have never even wondered how dict works it seems obvious that this is how it should work.
Yes, Ruby also has the same self-imposed constraint on insertion order. I agree it makes it more complicated to change the hash table implementation but it seems it wouldn’t be impossible to adapt Swiss Tables to support this. For example, by storing the “insertion number” (an incrementing integer) in each bucket, and possibly using table groups (as Go does) so that you only need to sort the buckets in a group, not the whole table, in order to yield entries in order. But this is just a naive sketch and I am not an expert on hash tables!
ihtabs preserve iteration order and have performance competitive with swiss tables (while not requiring as high a load factor): https://github.com/vnmakarov/ihtab
Comments
Python's hash table type, dict, promises that it remembers insertion order. If you put the mapping 5 => "dog" in first, then when we ask what is in the dict we'd get told 5 => "dog" first, duh.
Historically Python had a more conventional but very, very badly implemented hash table type, the "I can't believe it can sort"† of hash tables. Somebody wanted a hash table type which remembers insertion order because Python programmers have a bad habit of writing "Golden tests" in which that order matters even though in a good modern hash table type it's not guaranteed, so they built one. But because the built-in hash table type was garbage, this new OrderedDict type was much faster and much smaller despite solving a more difficult problem.
For a little while it was unclear if Python would decide to rewrite their dict type to have decent performance or just embrace this new better alternative and then they decided that because it's beginner friendly they will just embrace the OrderedDict and require that this type has ordering.
However, a good hash table doesn't inherently have this property, and that goes for the Swiss Table the same as other common designs. So they can't swap dict out for a Swiss Table without breaking their own promise that the dict type preserves insertion order.
If you're used to a language where this doesn't happen such as Rust, or C++ or Java or any number of other programming languages, that insertion ordering rule seems crazy, but if you've never used a programming language at all before and have never even wondered how dict works it seems obvious that this is how it should work.
† https://hectorcorrea.com/blog/2022-08-30/i-can-t-believe-it-...
Yes, Ruby also has the same self-imposed constraint on insertion order. I agree it makes it more complicated to change the hash table implementation but it seems it wouldn’t be impossible to adapt Swiss Tables to support this. For example, by storing the “insertion number” (an incrementing integer) in each bucket, and possibly using table groups (as Go does) so that you only need to sort the buckets in a group, not the whole table, in order to yield entries in order. But this is just a naive sketch and I am not an expert on hash tables!
ihtabs preserve iteration order and have performance competitive with swiss tables (while not requiring as high a load factor): https://github.com/vnmakarov/ihtab
That is what they claim but I'm very sceptical - not that they can preserve order but that they get similar performance.