All you need to store this kind of data is a simple file.
You mostly likely need a WAL Mode SQLite database. Most of the time, it's way simpler that handling state handling in concurrent situations yourself. (also, bindings are often available - if not outright bundled by default - under most common languages)
The "easy" way is all fun and games until your file is accessed in a concurrent fashion, and then your options are:
a) flat out die when concurrent things happens (file locks; default sqlite3 behaviour);
b) just write blindly to file and pretend concurrecy doesn't exist, but randomly lose data - (write to files directly like a crazy person; sqlite3 PRAGMA schema.synchronous = OFF)
c) allow reading at anytime, but serialize writing somehow (file locks + write + move atomic file operations; append writing and a journal; sqlite3 PRAGMA journal_mode=WAL)
You mostly likely need a WAL Mode SQLite database.
Overkill, If you are writing to the text file only to increment a number of visitors, none of what you mentioned above is even required.
Create a "counter-hit" file for each visitor, count the number of files in the resources directory. Populate the master file with the file count and delete all the temporary files. Configure it on a crontab for say every three seconds.
Online now was even easier. IFrame refresh within the homepage. Upon refresh, populate the file with something like "online=true". Read the file every X, if the file modifcation hasn't been refreshed within 30 seconds, nerf the text file and mark that user as offline. psuedo-dynamic, but times were different then. I coded these in perl back in the day for my RTCW clan and how I miss it much.
Being a webmaster was art, an art that's now lost..
The RaQ line used Apache which definitely supported concurrent requests. We're talking about 1999 here. Maybe this isn't as obvious to people today looking back, but concurrent request handling was absolutely required from the beginning: consider how slow your clients are! One single client on a slow dial-up modem can't be allowed to drag the whole site down.
It will be a forking web server. But the CPU scheduler will schedule process A while B is opening the file and reading and that. While B is then increasing the counter, leading to multiple processes with "wrong" data.l racing for the writes.
As I remember it, you would flock the file. You didn’t care if another process blocked for the short period it was held. It is not like you were getting huge amounts of traffic.
It can be much simpler if you're willing to write a couple lines of assembler: use an 8-byte file that contains one 64-bit counter, mmap() it, mlock() it, and use atomic CPU instructions to increment and read it.
Great question: it does not! It just removes a potential source of stalls.
If the page isn't resident in the page cache, the thread(s) executing the atomic increment will take a page fault and be blocked until the file data is read from storage. The latency of the fault might be significant, but the counter will remain accurate.
There's an important caveat I should have added: the counter file might be very stale after a power failure. If you care about the counter integrity, you have to msync(MS_SYNC) periodically, and that's expensive. It might actually stall all threads interacting with the page, depending on the filesystem; that used to be true but I'm not certain it still is (see https://lwn.net/Articles/486311/). Where writeback is allowed to race with writes, whether you would be guaranteed the 8-byte value written back wasn't "torn" without explicitly blocking increments while syncing is also an interesting question if DMA is involved...
For a guest book, you don't need anything like that. A simple solution could be to create a file for each entry. It's append-only anyway.
For a view counter - yes, you can use a full blown database to concurrently increment a simple integer value; or, you could do it in a few lines of code by yourself too.
Still a huge overkill, and you need to properly set up persistence if you use it this way.
If you know that you'll need a cannon anyway, then sure, go for it, but when all we deal with are flies there's no reason to go that far. You could easily set up this kind of website on servers so simple that getting redis running on them would inflate their complexity quite considerably.
I think you can avoid the lock and all else, if all you need is a counter: Open a file in append mode (O_APPEND) and then write a byte for each visit. To get the count take the file size.
Of course you have to make sure to not run out of diskspace or max file size of the OS/filesystem (2GB on a 32bit system?)
Comments
You mostly likely need a WAL Mode SQLite database. Most of the time, it's way simpler that handling state handling in concurrent situations yourself. (also, bindings are often available - if not outright bundled by default - under most common languages)
The "easy" way is all fun and games until your file is accessed in a concurrent fashion, and then your options are:
a) flat out die when concurrent things happens (file locks; default sqlite3 behaviour);
b) just write blindly to file and pretend concurrecy doesn't exist, but randomly lose data - (write to files directly like a crazy person; sqlite3 PRAGMA schema.synchronous = OFF)
c) allow reading at anytime, but serialize writing somehow (file locks + write + move atomic file operations; append writing and a journal; sqlite3 PRAGMA journal_mode=WAL)
Overkill, If you are writing to the text file only to increment a number of visitors, none of what you mentioned above is even required.
Create a "counter-hit" file for each visitor, count the number of files in the resources directory. Populate the master file with the file count and delete all the temporary files. Configure it on a crontab for say every three seconds.
Online now was even easier. IFrame refresh within the homepage. Upon refresh, populate the file with something like "online=true". Read the file every X, if the file modifcation hasn't been refreshed within 30 seconds, nerf the text file and mark that user as offline. psuedo-dynamic, but times were different then. I coded these in perl back in the day for my RTCW clan and how I miss it much.
Being a webmaster was art, an art that's now lost..
64 Player Depot. I always had a fondness for Tram. Played CAL Main. Good times. v43
This is likely what was done. Nobody really cares if a hit counter on a web page loses a few updates.
Given it's a single-threaded CPU, there's a fair chance the web server isn't concurrent anyway.
Linux got proper POSIX thread support with NPTL only in 2002.
The RaQ line used Apache which definitely supported concurrent requests. We're talking about 1999 here. Maybe this isn't as obvious to people today looking back, but concurrent request handling was absolutely required from the beginning: consider how slow your clients are! One single client on a slow dial-up modem can't be allowed to drag the whole site down.
It will be a forking web server. But the CPU scheduler will schedule process A while B is opening the file and reading and that. While B is then increasing the counter, leading to multiple processes with "wrong" data.l racing for the writes.
As I remember it, you would flock the file. You didn’t care if another process blocked for the short period it was held. It is not like you were getting huge amounts of traffic.
It can be much simpler if you're willing to write a couple lines of assembler: use an 8-byte file that contains one 64-bit counter, mmap() it, mlock() it, and use atomic CPU instructions to increment and read it.
Genuine question: Does it have to be mlocked for this to work?
Great question: it does not! It just removes a potential source of stalls.
If the page isn't resident in the page cache, the thread(s) executing the atomic increment will take a page fault and be blocked until the file data is read from storage. The latency of the fault might be significant, but the counter will remain accurate.
There's an important caveat I should have added: the counter file might be very stale after a power failure. If you care about the counter integrity, you have to msync(MS_SYNC) periodically, and that's expensive. It might actually stall all threads interacting with the page, depending on the filesystem; that used to be true but I'm not certain it still is (see https://lwn.net/Articles/486311/). Where writeback is allowed to race with writes, whether you would be guaranteed the 8-byte value written back wasn't "torn" without explicitly blocking increments while syncing is also an interesting question if DMA is involved...
I am going to ask gpt 4 to implement this for me so I can see what it looks like. Amazing.
I have to go to dinner so this isn't finished, but this is most of the code to prove it works: https://gist.github.com/jcalvinowens/0d7a5c327d863fca7c84daa...
How did it go?
For a guest book, you don't need anything like that. A simple solution could be to create a file for each entry. It's append-only anyway.
For a view counter - yes, you can use a full blown database to concurrently increment a simple integer value; or, you could do it in a few lines of code by yourself too.
$ apt install redis-server
$ redis-cli INCR hit-count
Still a huge overkill, and you need to properly set up persistence if you use it this way.
If you know that you'll need a cannon anyway, then sure, go for it, but when all we deal with are flies there's no reason to go that far. You could easily set up this kind of website on servers so simple that getting redis running on them would inflate their complexity quite considerably.
I think you can avoid the lock and all else, if all you need is a counter: Open a file in append mode (O_APPEND) and then write a byte for each visit. To get the count take the file size.
Of course you have to make sure to not run out of diskspace or max file size of the OS/filesystem (2GB on a 32bit system?)
Sqlite?? MS Access for the win!
You can do (c) in python with flock
https://www.php.net/manual/en/function.flock.php
Or redis.
I think you mean journal mode off. Synchronous off is nothing to do with concurrency.
d) write to a temp file and atomically move it onto the read path. you will miss counts, but nothing will break.