We started with trying to create something like this when we started Dark (https://darklang.com). We ultimately moved away from this style of programming, as we couldn't figure out a way to make it feel "code-y" enough for people to want to write programs in it.
I'm super interested in what folks want to use this for?
Yeah I like the idea of visual programming but it's not gonna replace everyday dev work in my opinion.
I would like to see more apps for tablets that introduce visual programming into everyday workflows. If you haven't seen the work of Ink & Switch check out these essays on end user programming [0] and an app for basically working on ideas [1], combine these and you get a fluid interface with an easy way of doing some computation (the visual programming bit) which is something I would love to use.
But visual programming gets messy if you attempt to do something complex and I don't know if there is any way around this. This then is where there needs to be programs with strong types that can compose with each other, like the example explained in this post [2].
These concepts are all things which anyone using a computer could use, not something just resticted to people who are already "developers", and I think we have to get into that mindset for this type of software to become more successful.
In theory one could structure any program this way right? In practice I've seen it work mostly for "creative programming", like in Max/MSP [1] or Unity3d [2]. NiFi [3] seems to be successful too, but kinda niche (and looks very daunting for general, simple problems).
I like the idea of having programs structured as "producer" and "consumer" nodes, with nodes being as close to pure functions as possible. But it seems like the graphical representation indeed gets in the way.
Perhaps the idea could still work without the visualization part? Each node a single file, listing inputs (maybe some sinks too) at the top. Still a graph, but users wouldn't _necesarily_ need to visualize it all the time.
Visual programming works very well when you have data flowing, but it doesnt work when you have loops and branching. But it is nice to support graphs for some tasks, it really speeds up some processes.
but it doesnt work when you have loops and branching
We managed to make visual programming work well with loops [1] and more or less OK with branching [2]. The trick is to look at loops from a functional perspective (i.e. as iter/map), rather than from imperative.
I think the ultimate test is whether it can be self hosted. eg. can the program be used to create the program itself!? Or what problem does it solve?!?
Both loops and branching/dispatching are examples of control flow, which is essentially independent from the 'data flow' default of most "visual programming" approaches. You can actually show both in the same visualization, but only in a well-defined "sub-diagram" sense. That is, a "block" in a data-flow visualization can be defined by a flowchart (showing control flow), and vice versa.
The dual problem is that flowcharts (in the traditional, control-flow-only sense) cannot show fork-join parallelism or fixed point constructions, while both are readily expressible by data flow.
I like PraxisLive[1] because it combines both code and nodes. Reconnecting nodes in different ways it's ideal for experimenting with different implementations or trying out different orders when applying filters (which works both for audio and graphics effects). It also facilitates changing values in real time without stopping the program, which is perfect when developing time-based software.
I also think visual coding works for high level abstractions rather than low-level procedural style workflows. E.g. for ML a visual flow could be import some data, train a model, deploy model to cloud. Orange Data Mining is an example of visual programming at the right level of abstraction but stops short of including enough modules to be useful in common data science workflows:
The general approach seems to work well for NodeRed and Unity? I think it's good to get to a point where we can use layered infrastructure to experiment more easily with this paradigm. I have tried to build something for mobile, partially inspired by Roocky's Boots, but I guess my assumption that drawing would be more accessible for mobile than typing didn't really hold (see flowgrid.org if interested).
Does that mean the Dark in Action on the homepage is outdated? Or is the UI something that carried over from the original design? I know there are "code-y" elements in that video and all the auto-complete features that are demoed are fantastic.
No, it's pretty current. The "code" in Dark is textual (it's not bytes on disk, but it's very far from nodes and edges). Dark is intended to be very code-y - although we're not fully there just yet coding in Dark mostly much feels like coding (discounting having to use a different editor).
Comments
We started with trying to create something like this when we started Dark (https://darklang.com). We ultimately moved away from this style of programming, as we couldn't figure out a way to make it feel "code-y" enough for people to want to write programs in it.
I'm super interested in what folks want to use this for?
Yeah I like the idea of visual programming but it's not gonna replace everyday dev work in my opinion.
I would like to see more apps for tablets that introduce visual programming into everyday workflows. If you haven't seen the work of Ink & Switch check out these essays on end user programming [0] and an app for basically working on ideas [1], combine these and you get a fluid interface with an easy way of doing some computation (the visual programming bit) which is something I would love to use.
But visual programming gets messy if you attempt to do something complex and I don't know if there is any way around this. This then is where there needs to be programs with strong types that can compose with each other, like the example explained in this post [2].
These concepts are all things which anyone using a computer could use, not something just resticted to people who are already "developers", and I think we have to get into that mindset for this type of software to become more successful.
[0] https://www.inkandswitch.com/end-user-programming.html
[1] https://www.inkandswitch.com/muse-studio-for-ideas.html
[2] https://pchiusano.github.io/2013-09-10/type-systems-and-ux-e...
In theory one could structure any program this way right? In practice I've seen it work mostly for "creative programming", like in Max/MSP [1] or Unity3d [2]. NiFi [3] seems to be successful too, but kinda niche (and looks very daunting for general, simple problems).
I like the idea of having programs structured as "producer" and "consumer" nodes, with nodes being as close to pure functions as possible. But it seems like the graphical representation indeed gets in the way.
Perhaps the idea could still work without the visualization part? Each node a single file, listing inputs (maybe some sinks too) at the top. Still a graph, but users wouldn't _necesarily_ need to visualize it all the time.
1: https://cycling74.com/products/max
2: https://unity.com/shader-graph
3: https://nifi.apache.org/
Creator or litegraph here.
Visual programming works very well when you have data flowing, but it doesnt work when you have loops and branching. But it is nice to support graphs for some tasks, it really speeds up some processes.
We managed to make visual programming work well with loops [1] and more or less OK with branching [2]. The trick is to look at loops from a functional perspective (i.e. as iter/map), rather than from imperative.
[1] https://easymorph.com/learn/iterations.html
[2] https://easymorph.com/learn/conditional-workflows.html
I think the ultimate test is whether it can be self hosted. eg. can the program be used to create the program itself!? Or what problem does it solve?!?
Both loops and branching/dispatching are examples of control flow, which is essentially independent from the 'data flow' default of most "visual programming" approaches. You can actually show both in the same visualization, but only in a well-defined "sub-diagram" sense. That is, a "block" in a data-flow visualization can be defined by a flowchart (showing control flow), and vice versa.
The dual problem is that flowcharts (in the traditional, control-flow-only sense) cannot show fork-join parallelism or fixed point constructions, while both are readily expressible by data flow.
I like PraxisLive[1] because it combines both code and nodes. Reconnecting nodes in different ways it's ideal for experimenting with different implementations or trying out different orders when applying filters (which works both for audio and graphics effects). It also facilitates changing values in real time without stopping the program, which is perfect when developing time-based software.
[1] https://praxislive.org/
I also think visual coding works for high level abstractions rather than low-level procedural style workflows. E.g. for ML a visual flow could be import some data, train a model, deploy model to cloud. Orange Data Mining is an example of visual programming at the right level of abstraction but stops short of including enough modules to be useful in common data science workflows:
https://orange.biolab.si/
The general approach seems to work well for NodeRed and Unity? I think it's good to get to a point where we can use layered infrastructure to experiment more easily with this paradigm. I have tried to build something for mobile, partially inspired by Roocky's Boots, but I guess my assumption that drawing would be more accessible for mobile than typing didn't really hold (see flowgrid.org if interested).
Does that mean the Dark in Action on the homepage is outdated? Or is the UI something that carried over from the original design? I know there are "code-y" elements in that video and all the auto-complete features that are demoed are fantastic.
No, it's pretty current. The "code" in Dark is textual (it's not bytes on disk, but it's very far from nodes and edges). Dark is intended to be very code-y - although we're not fully there just yet coding in Dark mostly much feels like coding (discounting having to use a different editor).