This is not at all comparable to what the OP submitted and would be at least a many thousands of lines worth of effort.
What watchman is doing is doing inotify-as-a-service for you, in order for any program to use the Linux inotify API it needs to be running constantly so it can keep consuming events from the kernel, and in the case of the use case watchman fulfills, maintain an internal database of how the current state differs from various times in the past.
There's lots of very nasty edge cases you need to deal with. E.g. if you don't consume the kernel events fast enough (or the buffer is too small) new events will be dropped.
You'll get told about this by the kernel, but now the only way to re-build your in-memory representation to match reality is to recursively walk the FS with readdir(), stat() etc, but at the same time consume new inotify events and update your in-memory structures while doing both.
Needless to say this is a lot more complex than just using watchman off the shelf.
It's also pointless to optimize this area, watchman usually returns in 2-3 milliseconds even on huge trees, but because of bugs / misfeatures in the current git-side implementation it can take hundreds of milliseconds until we make sense of all of that. This is because git doesn't really "know" about watchman/inotify, it just uses it as a replacement for walking the FS, but on big repositories other stuff can still be expensive.
To OP: I highly encourage you to submit this to the Git mailing list for comment. I've been involved in this area (helped write the Perl hook) and there's people who'd be interested in having this be e.g. a part of git's contrib/ directory so it can be compiled with git.
Comments
I wonder what kind of gains could be seen by using a fully rust-native bundle with something like notify [1]
[1]: https://crates.io/crates/notify
This is not at all comparable to what the OP submitted and would be at least a many thousands of lines worth of effort.
What watchman is doing is doing inotify-as-a-service for you, in order for any program to use the Linux inotify API it needs to be running constantly so it can keep consuming events from the kernel, and in the case of the use case watchman fulfills, maintain an internal database of how the current state differs from various times in the past.
There's lots of very nasty edge cases you need to deal with. E.g. if you don't consume the kernel events fast enough (or the buffer is too small) new events will be dropped.
You'll get told about this by the kernel, but now the only way to re-build your in-memory representation to match reality is to recursively walk the FS with readdir(), stat() etc, but at the same time consume new inotify events and update your in-memory structures while doing both.
Needless to say this is a lot more complex than just using watchman off the shelf.
It's also pointless to optimize this area, watchman usually returns in 2-3 milliseconds even on huge trees, but because of bugs / misfeatures in the current git-side implementation it can take hundreds of milliseconds until we make sense of all of that. This is because git doesn't really "know" about watchman/inotify, it just uses it as a replacement for walking the FS, but on big repositories other stuff can still be expensive.
To OP: I highly encourage you to submit this to the Git mailing list for comment. I've been involved in this area (helped write the Perl hook) and there's people who'd be interested in having this be e.g. a part of git's contrib/ directory so it can be compiled with git.
Thanks for the explanation, indeed watchman does all the heavy lifting! I will submit it to the mailing list, I'd love to see it in git core.