Because hacker news doesn't use a database, the site is file driven and runs entirely out of ram. Changes are made in memory and logged to disk and upon server restart, the files are lazily loaded on demand to prevent the need to load the entire db all at once. This might sound crazy but it's actually faster (once loaded) and easier to program than a relational database.
As hacker news was a fun project for Paul, I'd imagine he doesn't want to touch a relational database with a ten foot pole; I don't blame him. You can get a long way without ever using a real database, files and in memory hash tables work fine up to a pretty decent amount of traffic. It only becomes problematic when you need more than one server.
Interesting; I have noticed that once the threads page loads, it generally works fine the second time. I've been trying not to hammer on it too much to see when it works/doesn't work, though.
I guess it's one way to tell if the server's been restarted since I last was here. :)
Comments
This is a different problem from the one we're experiencing now. Threads takes a long time the first time because items are lazily loaded.
Why lazily load items yourself instead of use a database, like Postgres or even SQLite, that handles the paging in and out itself?
Because hacker news doesn't use a database, the site is file driven and runs entirely out of ram. Changes are made in memory and logged to disk and upon server restart, the files are lazily loaded on demand to prevent the need to load the entire db all at once. This might sound crazy but it's actually faster (once loaded) and easier to program than a relational database.
As hacker news was a fun project for Paul, I'd imagine he doesn't want to touch a relational database with a ten foot pole; I don't blame him. You can get a long way without ever using a real database, files and in memory hash tables work fine up to a pretty decent amount of traffic. It only becomes problematic when you need more than one server.
Interesting; I have noticed that once the threads page loads, it generally works fine the second time. I've been trying not to hammer on it too much to see when it works/doesn't work, though.
I guess it's one way to tell if the server's been restarted since I last was here. :)