Yes, io_uring is significantly faster than epoll (I think I had like 20% faster req/s with io_uring.) The catch is that its kernel opt-in and disabled just about everywhere for security reasons. I think that it has direct memory sharing between the kernel and user-land which is kind of yikes. There's been multiple exploits that hit io_uring in recent times. It's because of this that even engineering projects that try to reach the highest performance possible (like Go) don't really bake io_uring in as a sane default. Though if you want to take the risk you can always run it yourself for your favourite language. It is faster but the cost is possible exploits.
The main reason why it gets disabled is fixed now, the latest RC got cBPF support and as such you can restrict what OPs can be run now instead of just fully disabling it.
Well the reason it's disabled now is the recent history of pretty bad vulnerabilities. It probably needs to go a while without new vulnerabilities before it makes sense to enable by default. It's pretty complex completely unsafe C code, after all.
With seccomp you can disable a single syscall which might be exploitable since you didn’t get around to patch your kernel, with cBPF you can now do the same for io_uring
Quite depends, I had times when my posix emulation of io_uring (with poll, not epoll) was faster than io_uring. For large zero-copy buffers, io_uring is king however. Also io_uring is useful even for non asynchronous IO as it can implement chain of operations as single atomic operation (mkdir + open it for example).
For something like networking, if you are maximizing packets per second, you'll hit kernel limits[1] very quickly and instead have to start leveraging features like GSO/GRO or completely bypass the network stack.
RHEL 9 and 10 now fully support io_uring by default. It is very recent, but this covers a lot of corporate Linux installs. Gemini 'said' Ubuntu and SuSE support it as well, but did not provide any links to prove it.
It's still seccomp'd off in most environments because io-uring is still a seccomp bypass that doesn't play well with kernel security systems (audit subsystem), even if it weren't also like the #1 or #2 exploit vector for privesc.
Their point is that that functionality is not available on older kernels (such as those in RHEL 9 and 10) and so most sandboxes will continue to block it outright for a while, though eventually one would expect Red Hat to backport it.
(We haven't even added support for the new cBPF io_uring stuff to low-level container runtimes like runc yet, though I did review the patchset on LKML earlier this year and planned to get working on it when I have time. But as it requires spec changes, expect it to take 6-12 months at best...)
For a project like Go, wouldn't it be an option to do one-time iouring feature detection in the runtime startup? Exploits are an issue for the entire OS, not the program choosing to use iouring, yeah?
The ring buffers are in shared memory not kernel private. The ring buffers (submission and completion) are shared between kernel space and user space. User publishes requests via submission queue entries (updates tail of buffer while kernel reads head of the buffer), kernel shifts the submission queue buffer on its end and returns a completion queue event by publishing to completion buffer. User pulls from this buffer (specifically the head, kernel updates tail of buffer) in user space.
Comments
Yes, io_uring is significantly faster than epoll (I think I had like 20% faster req/s with io_uring.) The catch is that its kernel opt-in and disabled just about everywhere for security reasons. I think that it has direct memory sharing between the kernel and user-land which is kind of yikes. There's been multiple exploits that hit io_uring in recent times. It's because of this that even engineering projects that try to reach the highest performance possible (like Go) don't really bake io_uring in as a sane default. Though if you want to take the risk you can always run it yourself for your favourite language. It is faster but the cost is possible exploits.
The main reason why it gets disabled is fixed now, the latest RC got cBPF support and as such you can restrict what OPs can be run now instead of just fully disabling it.
Well the reason it's disabled now is the recent history of pretty bad vulnerabilities. It probably needs to go a while without new vulnerabilities before it makes sense to enable by default. It's pretty complex completely unsafe C code, after all.
Then just disable that singular function which causes the issue instead of a whole kernel subsystem. The current release can do that.
What's the singular function that has a chance of turning out to be exploitable in the future?
With seccomp you can disable a single syscall which might be exploitable since you didn’t get around to patch your kernel, with cBPF you can now do the same for io_uring
And which single syscall might be exploitable? I don't wanna wait until after an exploit to disable it
Nothing stops you from disabling everything you don’t need and now io_uring offers that per OP too.
I don’t get what you mean, in the past it was all or nothing, now you can disable every single operation selectively.
Gotta read the usual sources to find out what you gotta disable on your machines, like you have to for every single syscall since forever.
I mean exactly what I said. How do I know which parts of io_uring won't have security vulnerabilities discovered in the future?
Yes, but what parts of io_uring are safe to not disable?
It's not complex actually, but it is C...
Quite depends, I had times when my posix emulation of io_uring (with poll, not epoll) was faster than io_uring. For large zero-copy buffers, io_uring is king however. Also io_uring is useful even for non asynchronous IO as it can implement chain of operations as single atomic operation (mkdir + open it for example).
For something like networking, if you are maximizing packets per second, you'll hit kernel limits[1] very quickly and instead have to start leveraging features like GSO/GRO or completely bypass the network stack.
1: https://github.com/axboe/liburing/discussions/1346
Also it’s nice for things like SPI which have no user space non-blocking API.
SPI the bus?
RHEL 9 and 10 now fully support io_uring by default. It is very recent, but this covers a lot of corporate Linux installs. Gemini 'said' Ubuntu and SuSE support it as well, but did not provide any links to prove it.
https://access.redhat.com/solutions/4723221
Go should reconsider support. They should have a 'go' at it.
It's still seccomp'd off in most environments because io-uring is still a seccomp bypass that doesn't play well with kernel security systems (audit subsystem), even if it weren't also like the #1 or #2 exploit vector for privesc.
That’s solved as of last week, you can use cBPF now to disable functionality.
How solved? AFAIK it's not meaningfully shipped but happy to hear otherwise.
you can now disable opertions with cBPF, like you would be able to with seccomp for normal syscalls.
Their point is that that functionality is not available on older kernels (such as those in RHEL 9 and 10) and so most sandboxes will continue to block it outright for a while, though eventually one would expect Red Hat to backport it.
(We haven't even added support for the new cBPF io_uring stuff to low-level container runtimes like runc yet, though I did review the patchset on LKML earlier this year and planned to get working on it when I have time. But as it requires spec changes, expect it to take 6-12 months at best...)
Thanks for that! Hope 2027 will then be the year we get the speed back all those spectre etc mitgitations ate for those syscalls io_uring can replace.
For a project like Go, wouldn't it be an option to do one-time iouring feature detection in the runtime startup? Exploits are an issue for the entire OS, not the program choosing to use iouring, yeah?
Any kind of poll mode networking:
Rdma, dpdk, io_uring it’s really kind of up to the user to do the memory isolation
In io_urings case tho, you can’t do much because the rings are in the kernel.
I’m hopeful though that with Llm things will get better.
But it’s just hard problem to solve . Very difficult to do in the kernel itself, and folks don’t really even understand tuning for it.
The ring buffers are in shared memory not kernel private. The ring buffers (submission and completion) are shared between kernel space and user space. User publishes requests via submission queue entries (updates tail of buffer while kernel reads head of the buffer), kernel shifts the submission queue buffer on its end and returns a completion queue event by publishing to completion buffer. User pulls from this buffer (specifically the head, kernel updates tail of buffer) in user space.