The boxes and lines approaches work very well when you're doing coordination programming tasks rather than algorithmic and numerical ones. Instead of forcing people to use rigidly specified academic / formal systems like UML sequence and activity diagrams, Visio has withstood the UX of most business users' needs that have little background in programming. A lot of what your system does seems to be a close analogue to what SRC Computers developed in its compilers to perform dataflow analysis with some auto-vectorization logic. However, a lot of the CPU-FPGA crossover works better with vector-centric languages like Matlab or Fortran than primarily scalar constructs like in C or Java.
One thing I can say with confidence is that general purpose visual programming languages are extremely difficult to produce due to UX / UI expectations, but for smaller companies significant progress can be made with domain specific visual languages which your product seems to be aiming at. Good job avoiding one of the tar pits of failed programming paradigms of the past century.
One thing I observed as an everyday user and developer on a visual language / ecosystem was that design patterns emerged that were really easy to do in most text programs that were really cumbersome to do in block diagrams but I could find almost nothing that was easier in a visual language than a textual one. For example, most modern class-based OOP systems have virtual inheritance possibilities where you can overload behaviors and we have compiler resolution that determines what can and can't perform the override. I have not found an easy way to show this visually across many block diagrams without introducing a lot of confusion (eg. dotted lines instead of solid ones, collapsing box groups together) because of issues like dynamic dispatch. This stuff is pretty easy to form the constructs as a competent programmer but once I was in a visual language I was flipping around pages and pages of pretty blocks that started to get in the way rather than guide me towards understanding the 500 blocks I dropped down and the rats nest of lines pointing everywhere with switch statements galore. In our system I tried to write dependency injection and other elements of aspect oriented programming in but the inability for easy to associate referential steps in our specific system really messed with usability and thus debugging capabilities. As a power user I was not that much more productive than a relative novice in our system, and that really bothered me on what kept my productivity in the system from growing.
Eventually it all came down to inability for automatic, sensible scoping of logical components to be performed in our software due to lack of awareness of what the user is actually looking for because of lack of typing (we didn't have data structures / types - numbers and strings were all the same raw data) - all the analysis that could be done effectively for users was done following program execution, not really static analysis of a dataflow / graph. Everything was a side effect in a block diagram system just like how it can be in OOP, but unlike in OOP I had a heck of a time pointing references to a fragment under test to, say, a mock step easily (we did add them in eventually but scoping the substitution was insane). We have automatic folding in text editors that does pretty well, too, but grouping a bunch of randomly laid out shapes on a canvas together and hiding them while selecting them with a mouse was really painful for the minimal value it'd provide.
What happens is that the structure of the dataflow becomes a concern orthogonal to the basic algorithmic steps like looping, string manipulation, logical operators, arithmetic operators. Because most of the limitations of software production happen due to people's lack of tooling or abilities to reason about their data and behaviors efficiently / effectively, this means that given the failures so far in visual programming systems we probably need to re-think usability first before trying to develop a programming system around it.
The reasons we were able to get much success in our product with a visual programming paradigm was
1. we used users' existing expectations from products like Visio to help with onboarding / training
2. we developed complex logic like web requests in normal code (Java) and packaged them up into block diagram components that are easy to configure, so it was a lot like Automator constructs
3. everything was fairly easy to trace during program runs (required for auditors anyway)
This all came with a large amount of resource costs though which is why it never really would have worked as a consumer product unfortunately.
Fantastic! Really insightful feedback. How to do sensible scoping of logical components has been something I've been trying to rethink recently. Currently, you can read through text and imagine the logical components. But it'd be interesting to see more discussion put into modes of visual expressions, especially when combined with more textual information.
What would you classify as easy to associate referential steps? I've been using html5 components to combine hierarchical structure while being able to still use manual id/herd to provide the occasional "slice" outside the normal data stack. However, it still requires modifying a textual code stack.
Also the aspect oriented programming part sounds interesting. Did you ever try having "behavioral subnodes" which modify the parent node? I found it to be powerful, but only when used in a hierarchical data flow pattern. Though I haven't built the gui side of that approach yet.
Thanks for the feedback. I'm working on a set of programming tools based on many of these ideas, and trying a few angles to visualized those orthogonal concepts in a more interactive way. Oh, and the resource cost does seem to be a blocker for consumer products in this area.
Comments
The boxes and lines approaches work very well when you're doing coordination programming tasks rather than algorithmic and numerical ones. Instead of forcing people to use rigidly specified academic / formal systems like UML sequence and activity diagrams, Visio has withstood the UX of most business users' needs that have little background in programming. A lot of what your system does seems to be a close analogue to what SRC Computers developed in its compilers to perform dataflow analysis with some auto-vectorization logic. However, a lot of the CPU-FPGA crossover works better with vector-centric languages like Matlab or Fortran than primarily scalar constructs like in C or Java.
One thing I can say with confidence is that general purpose visual programming languages are extremely difficult to produce due to UX / UI expectations, but for smaller companies significant progress can be made with domain specific visual languages which your product seems to be aiming at. Good job avoiding one of the tar pits of failed programming paradigms of the past century.
What UI/UX expectations make it difficult to make visual programming languages general purpose?
One thing I observed as an everyday user and developer on a visual language / ecosystem was that design patterns emerged that were really easy to do in most text programs that were really cumbersome to do in block diagrams but I could find almost nothing that was easier in a visual language than a textual one. For example, most modern class-based OOP systems have virtual inheritance possibilities where you can overload behaviors and we have compiler resolution that determines what can and can't perform the override. I have not found an easy way to show this visually across many block diagrams without introducing a lot of confusion (eg. dotted lines instead of solid ones, collapsing box groups together) because of issues like dynamic dispatch. This stuff is pretty easy to form the constructs as a competent programmer but once I was in a visual language I was flipping around pages and pages of pretty blocks that started to get in the way rather than guide me towards understanding the 500 blocks I dropped down and the rats nest of lines pointing everywhere with switch statements galore. In our system I tried to write dependency injection and other elements of aspect oriented programming in but the inability for easy to associate referential steps in our specific system really messed with usability and thus debugging capabilities. As a power user I was not that much more productive than a relative novice in our system, and that really bothered me on what kept my productivity in the system from growing.
Eventually it all came down to inability for automatic, sensible scoping of logical components to be performed in our software due to lack of awareness of what the user is actually looking for because of lack of typing (we didn't have data structures / types - numbers and strings were all the same raw data) - all the analysis that could be done effectively for users was done following program execution, not really static analysis of a dataflow / graph. Everything was a side effect in a block diagram system just like how it can be in OOP, but unlike in OOP I had a heck of a time pointing references to a fragment under test to, say, a mock step easily (we did add them in eventually but scoping the substitution was insane). We have automatic folding in text editors that does pretty well, too, but grouping a bunch of randomly laid out shapes on a canvas together and hiding them while selecting them with a mouse was really painful for the minimal value it'd provide.
What happens is that the structure of the dataflow becomes a concern orthogonal to the basic algorithmic steps like looping, string manipulation, logical operators, arithmetic operators. Because most of the limitations of software production happen due to people's lack of tooling or abilities to reason about their data and behaviors efficiently / effectively, this means that given the failures so far in visual programming systems we probably need to re-think usability first before trying to develop a programming system around it.
The reasons we were able to get much success in our product with a visual programming paradigm was 1. we used users' existing expectations from products like Visio to help with onboarding / training 2. we developed complex logic like web requests in normal code (Java) and packaged them up into block diagram components that are easy to configure, so it was a lot like Automator constructs 3. everything was fairly easy to trace during program runs (required for auditors anyway)
This all came with a large amount of resource costs though which is why it never really would have worked as a consumer product unfortunately.
Fantastic! Really insightful feedback. How to do sensible scoping of logical components has been something I've been trying to rethink recently. Currently, you can read through text and imagine the logical components. But it'd be interesting to see more discussion put into modes of visual expressions, especially when combined with more textual information.
What would you classify as easy to associate referential steps? I've been using html5 components to combine hierarchical structure while being able to still use manual id/herd to provide the occasional "slice" outside the normal data stack. However, it still requires modifying a textual code stack.
Also the aspect oriented programming part sounds interesting. Did you ever try having "behavioral subnodes" which modify the parent node? I found it to be powerful, but only when used in a hierarchical data flow pattern. Though I haven't built the gui side of that approach yet.
Thanks for the feedback. I'm working on a set of programming tools based on many of these ideas, and trying a few angles to visualized those orthogonal concepts in a more interactive way. Oh, and the resource cost does seem to be a blocker for consumer products in this area.
That was extremely interesting, thank you.