Yes I agree -- threads without shared state or with thread-safe shared data structures is a perfectly good programming paradigm (and pretty well proven too).
One of my pet ideas is just to implement a Go-like CSP with plain pthreads. Channels could be pipes of pointers. Channel select is just select(). No weird M:N runtime needed. I don't want an operating system in my programming language.
This is just stolen from Programming in Lua chapter 30 -- threads and states (
https://www.lua.org/pil/ , not in the 1st online edition unfortunately)
Lua has coroutines within the interpreter, but to utilize all the cores he suggests layering threads and multiple Lua interpreters on top.
One of my pet ideas is just to implement a Go-like CSP with plain pthreads. Channels could be pipes of pointers. Channel select is just select(). No weird M:N runtime needed.
Please do it! I feel like we've forgotten the lessons of NPTL, when everyone tried M:N and collectively came to a consensus that M:N wasn't worth it in practice.
OK :) I actually have a reasonable project to do it on... I'm implementing a new shell, which is making decent progress, and I've been documenting some interesting things here:
It's very compatible with bash so I think it has a chance of being adopted. And I would like to add structured data pipelines, which powershell has. I was thinking of implementing it with the threads and pipes of pointers scheme (and probably the condition variable).
I guess the shell concurrency model is more like a subset of CSP, but the more general CSP model seems useful and fairly easily implementable. I feel like it should be like 200 lines of code, so I should try it sooner rather than later. Just start porting some simple Go programs to it.
I've just spent 1h at work (silly me) reading your blog, it is illuminating! I love your writing style, and I can't wait to see the your shell in action! Just the full-parse-before-execution is well worth it, especially when deploying scripts on servers.
If you open source the project, I'd love to try giving you a hand. Anyway, good luck with the project!
I'm rounding the corner on parsing hundreds of thousands of lines of bash scripts now... the prototype is in Python as mentioned in the first post, and the executor isn't complete, but if you want the parse-before-execution, that is working well.
ShellCheck does exist though. IIRC I had mixed experience with it -- it did actually find one bug, but on the other hand it spewed hundreds of warnings about double quoting vars, which is technically true, but not the best use of time for most scripts I write. I'd rather just get rid of stupid quoting rules, which is one of the #1 priorities.
(As far as writing, I find that "omit needless words" from Strunk & White goes a long way. Words like "very" and "a little" somehow spray themselves all over my writing; they are rarely useful and I kill them on editing passes :) )
That's probably a good idea. However it will most likely not work out with pipes as I don't know how you could implement the synchronous (unbuffered) channels with them.
IMHO unbuffered channels are one of the most powerful constructs in Go, since they guarantee that "resources" are always on one-side of the channel and are taken care of, and never stored in a channel (or promise, ...) where they could get abondoned/lost. It also allows to make some other assumptions like "the in-memory server has taken my request through a channel and is now working on it and will answer through a chnanel soon" or "the server has shut down so I can't write to the channel", but never "the write to the channel succeeded but nobody cares about it".
Yeah actually the Lua implementation I mentioned uses a condition variable (not sure about the mutex).
The unbuffered channel would be a good reason to use that scheme. I was thinking of using pipes so Go's select reduces to the select() system call. But I think you can just do both -- it's cheap. Write to a pipe and and notify a condition variable. I'll experiment with it.
Comments
Yes I agree -- threads without shared state or with thread-safe shared data structures is a perfectly good programming paradigm (and pretty well proven too).
One of my pet ideas is just to implement a Go-like CSP with plain pthreads. Channels could be pipes of pointers. Channel select is just select(). No weird M:N runtime needed. I don't want an operating system in my programming language.
This is just stolen from Programming in Lua chapter 30 -- threads and states ( https://www.lua.org/pil/ , not in the 1st online edition unfortunately)
Lua has coroutines within the interpreter, but to utilize all the cores he suggests layering threads and multiple Lua interpreters on top.
Please do it! I feel like we've forgotten the lessons of NPTL, when everyone tried M:N and collectively came to a consensus that M:N wasn't worth it in practice.
OK :) I actually have a reasonable project to do it on... I'm implementing a new shell, which is making decent progress, and I've been documenting some interesting things here:
http://www.oilshell.org/blog/
It's very compatible with bash so I think it has a chance of being adopted. And I would like to add structured data pipelines, which powershell has. I was thinking of implementing it with the threads and pipes of pointers scheme (and probably the condition variable).
I guess the shell concurrency model is more like a subset of CSP, but the more general CSP model seems useful and fairly easily implementable. I feel like it should be like 200 lines of code, so I should try it sooner rather than later. Just start porting some simple Go programs to it.
( My last post about parsing expressions got buried on HN but I think it is fairly interesting to a specialized audience: http://www.oilshell.org/blog/2016/11/01.html )
I've just spent 1h at work (silly me) reading your blog, it is illuminating! I love your writing style, and I can't wait to see the your shell in action! Just the full-parse-before-execution is well worth it, especially when deploying scripts on servers.
If you open source the project, I'd love to try giving you a hand. Anyway, good luck with the project!
Great thanks! Yes it will be open source.
I'm rounding the corner on parsing hundreds of thousands of lines of bash scripts now... the prototype is in Python as mentioned in the first post, and the executor isn't complete, but if you want the parse-before-execution, that is working well.
ShellCheck does exist though. IIRC I had mixed experience with it -- it did actually find one bug, but on the other hand it spewed hundreds of warnings about double quoting vars, which is technically true, but not the best use of time for most scripts I write. I'd rather just get rid of stupid quoting rules, which is one of the #1 priorities.
(As far as writing, I find that "omit needless words" from Strunk & White goes a long way. Words like "very" and "a little" somehow spray themselves all over my writing; they are rarely useful and I kill them on editing passes :) )
That's probably a good idea. However it will most likely not work out with pipes as I don't know how you could implement the synchronous (unbuffered) channels with them.
IMHO unbuffered channels are one of the most powerful constructs in Go, since they guarantee that "resources" are always on one-side of the channel and are taken care of, and never stored in a channel (or promise, ...) where they could get abondoned/lost. It also allows to make some other assumptions like "the in-memory server has taken my request through a channel and is now working on it and will answer through a chnanel soon" or "the server has shut down so I can't write to the channel", but never "the write to the channel succeeded but nobody cares about it".
Easy. Just implement them with a mutex and a condvar. If you want, layer some lock free algorithm on top for the fast path.
Yeah actually the Lua implementation I mentioned uses a condition variable (not sure about the mutex).
The unbuffered channel would be a good reason to use that scheme. I was thinking of using pipes so Go's select reduces to the select() system call. But I think you can just do both -- it's cheap. Write to a pipe and and notify a condition variable. I'll experiment with it.