I've written a [proprietary, though I have permission to open source it] `tailfhttpd`, which is a tiny, trivial HTTP/1.1 server that supports only `HEAD`s and `GET`s of regular files, but with a twist:
- it supports `ETag`s, with ETags derived from
a file's st_dev, st_ino, and inode generation
number
- it supports setting *some* response headers via
xattrs (e.g., ETag, Content-Type, Vary,
Cache-Control, etc.)
- it supports conditional requests (i.e.,
`If-Match:`, `If-None-Match:`,
`If-Modified-Since:`)
- it supports `Range:` requests
- for `Range: bytes=${offset}-` `GET`s
the response does not finish (i.e.,
final chunk is not sent) until one of
- the file is unlinked
- the file is renamed
- the server is terminated
using inotify to find out about file
unlinks/renames
It does this using `epoll`, `inotify`, and `sendfile()`, with multiple fully-evented, async-I/O-using processes, each process being single-threaded. It is written in C in continuation passing style (CPS) with tiny continuations, so its memory footprint per client is also tiny. As a result it is blazingly fast, though it needs to be fronted with a reverse proxy for HTTPS (e.g., Nginx, Envoy, ...), sadly, but maybe I could teach it to use kssl.
I use it for tailing logs remotely, naturally, and as a poor man's Kafka. Between regular file byte offsets, ETags, and conditional requests one can build a reliable event publication system with this `tailfhttpd`. For example, and event stream can name the next instance ({local-part, ETag}) then be renamed out of the way to end in-progress `GET`s, and clients can resume from the new file.
With a few changes it could "tail" (watch) directories, and even allow `POST`ing events (which could be done by writing to a pipe the reader of which routes events to files that get served by `tailfhttpd`).
Because `tailfhttpd` just serves files, and because of the ETag thing, conditional requests, and xattrs, it's very easy to build more complex systems on top of it -- even shell scripts will suffice.
This chunked-encoding, "hanging-GET" thing is so unreasonably effective and cheap that I'm surprised how few systems support it.
I've visions of rewriting it in Rust and supporting H2 and especially H3/QUIC to reduce the per-client load (think of TCP TCBs and buffers) even more, and using io_uring instead of epoll for even better performance.
Oh, and this approach is fully standards-compliant. It's just a chunked-encoding, indefinite-end ("hanging") GET with all the relevant (but optional) behaviors (ETags, conditional requests, range requests, even the right end of the byte-range being left unspecified is within spec!).
Comments
I've written a [proprietary, though I have permission to open source it] `tailfhttpd`, which is a tiny, trivial HTTP/1.1 server that supports only `HEAD`s and `GET`s of regular files, but with a twist:
It does this using `epoll`, `inotify`, and `sendfile()`, with multiple fully-evented, async-I/O-using processes, each process being single-threaded. It is written in C in continuation passing style (CPS) with tiny continuations, so its memory footprint per client is also tiny. As a result it is blazingly fast, though it needs to be fronted with a reverse proxy for HTTPS (e.g., Nginx, Envoy, ...), sadly, but maybe I could teach it to use kssl.I use it for tailing logs remotely, naturally, and as a poor man's Kafka. Between regular file byte offsets, ETags, and conditional requests one can build a reliable event publication system with this `tailfhttpd`. For example, and event stream can name the next instance ({local-part, ETag}) then be renamed out of the way to end in-progress `GET`s, and clients can resume from the new file.
With a few changes it could "tail" (watch) directories, and even allow `POST`ing events (which could be done by writing to a pipe the reader of which routes events to files that get served by `tailfhttpd`).
Because `tailfhttpd` just serves files, and because of the ETag thing, conditional requests, and xattrs, it's very easy to build more complex systems on top of it -- even shell scripts will suffice.
This chunked-encoding, "hanging-GET" thing is so unreasonably effective and cheap that I'm surprised how few systems support it.
I've visions of rewriting it in Rust and supporting H2 and especially H3/QUIC to reduce the per-client load (think of TCP TCBs and buffers) even more, and using io_uring instead of epoll for even better performance.
Oh, and this approach is fully standards-compliant. It's just a chunked-encoding, indefinite-end ("hanging") GET with all the relevant (but optional) behaviors (ETags, conditional requests, range requests, even the right end of the byte-range being left unspecified is within spec!).