To turn this question on its head, why use multiple processes? That's not just rhetorical, I don't understand why you'd ask that question. The main advantages of separate processes are (1) any global variables are not shared between them, which is sometimes what you want (like Python's GIL), and (2) there is memory protection. Neither of those seem relevant to a Rust data processing program.
To actually answer your question a bit: Threads allow all the benefits of processes (except the two I just mentioned), with the added benefit of being lower overhead and allow sharing information more easily (but you can still use full-blown IPC to communicate between them if you wish).
I want the whole thing to come down. To me, that's preferable to a process silently crashing. Worse still, Linux's OOM killer can kill anything it wants. This is all fine if your processes are truly independent, but if they're not then you've not really gained much by using processes.
processes do provide well defined boundaries that can guarantee the integrity of each component separately, so it is realistic to be able to design an application that can survive the loss of one component even in a memory unsafe language.
Comments
To turn this question on its head, why use multiple processes? That's not just rhetorical, I don't understand why you'd ask that question. The main advantages of separate processes are (1) any global variables are not shared between them, which is sometimes what you want (like Python's GIL), and (2) there is memory protection. Neither of those seem relevant to a Rust data processing program.
To actually answer your question a bit: Threads allow all the benefits of processes (except the two I just mentioned), with the added benefit of being lower overhead and allow sharing information more easily (but you can still use full-blown IPC to communicate between them if you wish).
Not all the benefits.
Threads also introduce security bugs and possible instability (one thread crash brings the whole thing down).
I want the whole thing to come down. To me, that's preferable to a process silently crashing. Worse still, Linux's OOM killer can kill anything it wants. This is all fine if your processes are truly independent, but if they're not then you've not really gained much by using processes.
processes do provide well defined boundaries that can guarantee the integrity of each component separately, so it is realistic to be able to design an application that can survive the loss of one component even in a memory unsafe language.
I didn't say "all the benefits", I said
One of those two exceptions was indeed memory protection.
Fair enough.