Skip to content

Comment on Strategies for Fast Lexers

Comments

As introduced in the previous chapters, all identifers are hashed, thus we can also hash the known keywords at startup and make comparing them very fast.

One trick that postgres uses [1][2] is perfect hashing [3]. Since you know in advance what your keywords are, you can design such hashing functions that for each w(i) in list of i keywords W, h(w(i)) = i. It essentially means no collisions and it's O(i) for the memory requirement.

[1] https://github.com/postgres/postgres/blob/master/src/tools/P...

[2] https://github.com/postgres/postgres/blob/master/src/tools/g...

[3] https://en.wikipedia.org/wiki/Perfect_hash_function

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.