I started programming in LISP way back in 1971 on a Univac 1108
mainframe and also implemented a 68000-based Lisp system (~50K lines
of real-time assembly) for mobile-robotics use in 1983 - and so know
my way around the language.
All Lisp (or Smalltalk, or...) success stories I've read hinge on someone with an enormous amount of experience with the language. I'd argue that someone with that much experience could get the job done in (almost) any language. I'm surprised that someone with that much experience would put it down to language choice rather than deep knowledge of the problem domain.
That is close to saying that all languages are equivalent given the same amount of experience. I don't buy that. For a specific task some languages may be better suited / more productive than other languages, even given equivalent levels of experience.
I don't believe in a global linear scale of language power ("the blub theory"), but I do believe that some language may be better than others for a specific task given specific constraints.
E.g. if I have equivalent levels of experience in C++ and Python, I'm pretty sure I can write small webapp quicker in Python. OTOH if the languages are very similar, like Python and Ruby, the level of experience is much more important that the relative strengths and weaknesses of the languages.
Of course, it is seldom that you get that kind of fair comparison between two languages - usually everyone has a favorite language they know better than any other.
In some place I read that differences in languages could account for at most 30% of improvements in speed of development. This is an epsilon if compared to differences between programmers.
Another issue is that, if you are talking about small-to-medium sized applications then clearly there is a difference in languages. For example, it is pretty clear that writing a script is easier in perl than in C, or writing a medium sized expert system is easier in LISP than in Pascal.
However, if you consider large scale applications (100k+ LOC), then I don't believe there is any difference between writing in C, C++, Java, or Common LISP, as long as the programmer(s) have deep experience with the used language.
Just notice that the language is not going to solve the large-scale problem by itself. As long as such language has tools for creating abstractions, the code will have about the same complexity no matter what. If that complexity will be encapsulated in simple concepts (structs and functions) or higher level concepts (closures and continuation) depends on the taste of the developers and the language used.
30% is an understatement. Think of the productivity gain when you go from assembly to C. Now, consider the fact (I do mean fact) that the jump from assembly to C and the jump from C to LISP are comparable.
The choice of abstraction do matter. If you use weak ones, your productivity is taking a serious hit: your program will be bigger, more complex, and have more errors (squared).
C++ abstractions, for instance, are incredibly weak. Take the function abstraction, which isn't even complete: you have no way to write anonymous functions the way you write literal integers[1]. Higher level concepts, as you call them, aren't more complicated than the "simple" ones. Often, they are just less familiar and more consistent.
[1]: Anonymous functions should actually be called "literal functions":
(fun x -> 7 x + 42) -- a literal function
357 -- a literal integer
2 + 3 -- expression which yields a integer
f . g -- expression which yields a function
Nothing "high order" about that. This is just acknowledging that functions are mathematical objects like any other.
Funny, I went to a talk today on statistical methods for opinion analysis, and in the annotated corpus presented, the only opinion word that was used more often in a subjective frame than an objective frame is the word "fact".
There are two kinds of design patterns: architectural patterns (e.g. MVC) and language patterns (e.g. Iterator). In the language I use for work, C#, we have to use a lot of both kinds of patterns. Often the "all code must be in a closure or a method" way the language works gets in the way (I can only imagine it's much worse in Java). In 100k LOC, I bet 25-30% of it (random "from the gut" guess) must be language patterns (e.g. visitor).
When you realize that nearly none (if any at all) of the language patterns are needed in Lisp you realize that in 100k LOC you only work with the problem. It seems to happen to me pretty often that I run into situation where there are two possible representations of something, both having problems and I realize that in Lisp I wouldn't have even noticed the situation at all because I could have used a more natural solution right from the start (CLOS' generic function approach makes all the difference in the world here).
Keep in mind I've used C++ and its descendants longer than I've used lisp.
I've been feeling quite despondent about the general language/framework fanboyism on Hacker News. This comment is a breath of fresh air. Thank you.
I've worked on heavily used programs in several languages (including Lisp) and frankly in my experience the quality of the programmer is way more important than the framework they start from.
I am starting to arrive at my own conclusion to language power which is that if you already know the "$1,000,000 concepts" embedded in the language, the only things the language itself offers above that are dev environment and ecosystem improvements - better debugging features and tools, more libraries, more solutions to corner case problems, platform deployment options, etc.
So features that are mostly implementation-dependent, like incremental development, static type checking, contracts, CPAN/gem/easy_install type code repositories, reference books and mailing list discussions, etc. are the things that give a language value. Exposing powerful key concepts(like common data structures, garbage collection, reflection, macro systems...) directly through the language is also a win but not something impossible to work around.
To paraphrase a oft-abused quote, "every sufficiently complex program contains a Lisp." But that doesn't mean the conclusion is "start with Lisp, since you'll end up with one anyway." It may be that Ruby (for example) has a really cool library you want to use for a core part of your app, so you start writing the code in Ruby, and things are good and you make progress....and once you come across a problem requiring Lisp-type power, just by knowing how the solution would work in Lisp, you can usually devise a worse-is-better 80% solution that is right for your specific application. The final result may not be beautiful or pure, but it lets you do things incrementally without the up-front misgivings of "it would have saved so much time to start in Ruby..."
tl;dr: Learn languages to learn concepts, but build applications in an "environmentally-friendly" way. Perceived "potential power" is not a good reason to sweat and strain to build apps in your favorite language when you could go over to the current "industry standard" and save yourself months of effort.
On the other hand, nobody has an omniscient view of all languages and environments, hence people tend to stick with what they know....
That is close to saying that all languages are equivalent given the same amount of experience.
Not quite - there are a bunch of languages that are "up there" in terms of expressivity: Lisp, Smalltalk, Haskell, ML, etc. There's no data to say "learning Lisp will give you an enormous productivity boost". There is only data to say "people with 30 years experience in Lisp/Smalltalk/whatever tend to be very good programmers". And, well, duh.
Also, if he had to do it in C, say, could he really not? Even if he needed to invoke Greenspun's Law in the process...
Even if he needed to invoke Greenspun's Law in the process...
The question is whether that always happens for a complex system. Because if it does, then every sufficiently advanced programmer is a lisp programmer.
I recently (~5 months ago) started using Clojure heavily and now find myself much more productive using it than using any of the languages that I have considerably more experience with (I used C++ almost exclusively for 4 years for hobby (mostly game related) projects and toy virtual machines/interpreters; Python for ~4 years for prototyping, web development and GUI development; Java for 2.5 years for server (non-web) development - total programming time including hobby, uni and professional = ~10 years).
As an example, I'm very much interested in compilers, virtual machines, programming language design and such, so have been plying around with these in various languages over the years. I wrote my first toy language in VB; wrote a few interpreters, virtual machines and simple parsers in C++ and some parsers, interpreters and assembly code generators in Python; for uni, I implemented a parser and code generator (instruction selection, using maximal munch) in JavaCC - recently I wrote an assembly code generator in Clojure. It took me a weekend and it surpassed the power of anything[1] I'd written in VB, C++, Python or Java. Its flexible and can be easily be extended, its pretty smart and can do some basic optimisation (caching data in registers, function inlining...). I tried writing something similar in Python before, but gave up.
My point is that I have much much deeper knowledge of C++, Python and Java, yet I was able to build something MORE complex and extensible in Clojure, in less time, even though I'd only been using it for a few months.
Yes, my previous experience played some part in this, but I attribute most of it to the flexibility and power Lisp provides me through easy to use and powerful abstractions, flexible and convenient syntax and interactive development.
[1] Of the same scale - any interpreter, code generator, compiler etc. I've written much more powerful systems, of course, but they are huge in comparison and took a lot lot longer than a weekend to write.
I would say that he has an enormous amount of experience with several languages. He lists on his webpage: "Programming languages used over the past 39-years include C/C++, Common Lisp/CLOS/MOP (and other Lisp varients), ALGOL, FORTRAN, PL/1, SNOBOL, numerous assembly languages, and others."
So, he probably could have written it in C++ (or SNOBOL ;-) ), but he chose to do it in Lisp.
No, he could not have (short of bootstraping a Lisp system from C++). I, for one, have much more C++ experience than Ocaml experience. I am far more efficient with Ocaml: typically, my Ocaml programs are 3 to 5 times shorter, takes half the time to program, and almost no time to debug.
I have 10 years of C++ experience, and 1 year of part-time Python experience. I needed to create a simple graph algorithm to find maximum Hamiltonian paths. It took me 1hr to get it running in Python and 6 hours to translate it to C++.
It is amazing how much Python (and other high level languages such as lisp, Ocaml) allow you to focus on the problem you are solving not the irrelevant details of the solution.
All things being equal, lisp was in use much more back in the 70s and 80s than it was in the 90s or 00s. It makes sense for the majority of people who would think to use lisp to be a bit grayer.
But you have seen the article that we are talking about? It is just such an output where the software is written in Lisp and the result is sold commercially.
People have asked Jack to explain why he used Lisp for an embedded application. You can read the result in the linked article.
The guy who has posted the story also has written a bunch of Lisp based internet applications. If you look among the hardcore Common Lisp users, quite a lot are writing software. The once that are blogging superficial stuff not so much. Read the stuff on planet.lisp.org - that's usually quite useful.
Why do you assume people don't release software in Common Lisp?
When you buy a piece of no one writes on the packaging "we did this in C++!" or "We did this in Java!" Why would you expect a big "Common Lisp!" sticker on the packaging? The deliverable is machine code, not source code. The customer doesn't care what the source is written in.
Because people don't write C++ or Java or Python "success stories". When someone ships an app in Lisp, they blog about it and it gets posted here. If Lisp were widely used, it would not be newsworthy.
I don't think you hear much about the 'real' Lisp apps. I have for example never seen anything about Amazon's use of Lisp. But they use it. Did you ever read something about the applications that schedule airport operations? Or did you ever hear about SISCOG's software ( http://www.siscog.com/ ) who develop most of their software in Lisp? Their products are used to schedule the crews of public transport systems (trains) in major cities and states.
Comments
This is the key point:
I started programming in LISP way back in 1971 on a Univac 1108 mainframe and also implemented a 68000-based Lisp system (~50K lines of real-time assembly) for mobile-robotics use in 1983 - and so know my way around the language.
All Lisp (or Smalltalk, or...) success stories I've read hinge on someone with an enormous amount of experience with the language. I'd argue that someone with that much experience could get the job done in (almost) any language. I'm surprised that someone with that much experience would put it down to language choice rather than deep knowledge of the problem domain.
That is close to saying that all languages are equivalent given the same amount of experience. I don't buy that. For a specific task some languages may be better suited / more productive than other languages, even given equivalent levels of experience.
I don't believe in a global linear scale of language power ("the blub theory"), but I do believe that some language may be better than others for a specific task given specific constraints.
E.g. if I have equivalent levels of experience in C++ and Python, I'm pretty sure I can write small webapp quicker in Python. OTOH if the languages are very similar, like Python and Ruby, the level of experience is much more important that the relative strengths and weaknesses of the languages.
Of course, it is seldom that you get that kind of fair comparison between two languages - usually everyone has a favorite language they know better than any other.
In some place I read that differences in languages could account for at most 30% of improvements in speed of development. This is an epsilon if compared to differences between programmers.
Another issue is that, if you are talking about small-to-medium sized applications then clearly there is a difference in languages. For example, it is pretty clear that writing a script is easier in perl than in C, or writing a medium sized expert system is easier in LISP than in Pascal.
However, if you consider large scale applications (100k+ LOC), then I don't believe there is any difference between writing in C, C++, Java, or Common LISP, as long as the programmer(s) have deep experience with the used language.
Just notice that the language is not going to solve the large-scale problem by itself. As long as such language has tools for creating abstractions, the code will have about the same complexity no matter what. If that complexity will be encapsulated in simple concepts (structs and functions) or higher level concepts (closures and continuation) depends on the taste of the developers and the language used.
30% is an understatement. Think of the productivity gain when you go from assembly to C. Now, consider the fact (I do mean fact) that the jump from assembly to C and the jump from C to LISP are comparable.
The choice of abstraction do matter. If you use weak ones, your productivity is taking a serious hit: your program will be bigger, more complex, and have more errors (squared).
C++ abstractions, for instance, are incredibly weak. Take the function abstraction, which isn't even complete: you have no way to write anonymous functions the way you write literal integers[1]. Higher level concepts, as you call them, aren't more complicated than the "simple" ones. Often, they are just less familiar and more consistent.
[1]: Anonymous functions should actually be called "literal functions":
Nothing "high order" about that. This is just acknowledging that functions are mathematical objects like any other.Funny, I went to a talk today on statistical methods for opinion analysis, and in the annotated corpus presented, the only opinion word that was used more often in a subjective frame than an objective frame is the word "fact".
I just don't find this to be the case.
There are two kinds of design patterns: architectural patterns (e.g. MVC) and language patterns (e.g. Iterator). In the language I use for work, C#, we have to use a lot of both kinds of patterns. Often the "all code must be in a closure or a method" way the language works gets in the way (I can only imagine it's much worse in Java). In 100k LOC, I bet 25-30% of it (random "from the gut" guess) must be language patterns (e.g. visitor).
When you realize that nearly none (if any at all) of the language patterns are needed in Lisp you realize that in 100k LOC you only work with the problem. It seems to happen to me pretty often that I run into situation where there are two possible representations of something, both having problems and I realize that in Lisp I wouldn't have even noticed the situation at all because I could have used a more natural solution right from the start (CLOS' generic function approach makes all the difference in the world here).
Keep in mind I've used C++ and its descendants longer than I've used lisp.
I've been feeling quite despondent about the general language/framework fanboyism on Hacker News. This comment is a breath of fresh air. Thank you.
I've worked on heavily used programs in several languages (including Lisp) and frankly in my experience the quality of the programmer is way more important than the framework they start from.
Can you provide a source for the 30% figure?
Do you have personal experience writing a 100k+ LOC application in two or more of the languages C, Java and Lisp?
I am starting to arrive at my own conclusion to language power which is that if you already know the "$1,000,000 concepts" embedded in the language, the only things the language itself offers above that are dev environment and ecosystem improvements - better debugging features and tools, more libraries, more solutions to corner case problems, platform deployment options, etc.
So features that are mostly implementation-dependent, like incremental development, static type checking, contracts, CPAN/gem/easy_install type code repositories, reference books and mailing list discussions, etc. are the things that give a language value. Exposing powerful key concepts(like common data structures, garbage collection, reflection, macro systems...) directly through the language is also a win but not something impossible to work around.
To paraphrase a oft-abused quote, "every sufficiently complex program contains a Lisp." But that doesn't mean the conclusion is "start with Lisp, since you'll end up with one anyway." It may be that Ruby (for example) has a really cool library you want to use for a core part of your app, so you start writing the code in Ruby, and things are good and you make progress....and once you come across a problem requiring Lisp-type power, just by knowing how the solution would work in Lisp, you can usually devise a worse-is-better 80% solution that is right for your specific application. The final result may not be beautiful or pure, but it lets you do things incrementally without the up-front misgivings of "it would have saved so much time to start in Ruby..."
tl;dr: Learn languages to learn concepts, but build applications in an "environmentally-friendly" way. Perceived "potential power" is not a good reason to sweat and strain to build apps in your favorite language when you could go over to the current "industry standard" and save yourself months of effort.
On the other hand, nobody has an omniscient view of all languages and environments, hence people tend to stick with what they know....
That is close to saying that all languages are equivalent given the same amount of experience.
Not quite - there are a bunch of languages that are "up there" in terms of expressivity: Lisp, Smalltalk, Haskell, ML, etc. There's no data to say "learning Lisp will give you an enormous productivity boost". There is only data to say "people with 30 years experience in Lisp/Smalltalk/whatever tend to be very good programmers". And, well, duh.
Also, if he had to do it in C, say, could he really not? Even if he needed to invoke Greenspun's Law in the process...
Even if he needed to invoke Greenspun's Law in the process...
The question is whether that always happens for a complex system. Because if it does, then every sufficiently advanced programmer is a lisp programmer.
Heres why I like Lisp:
I recently (~5 months ago) started using Clojure heavily and now find myself much more productive using it than using any of the languages that I have considerably more experience with (I used C++ almost exclusively for 4 years for hobby (mostly game related) projects and toy virtual machines/interpreters; Python for ~4 years for prototyping, web development and GUI development; Java for 2.5 years for server (non-web) development - total programming time including hobby, uni and professional = ~10 years).
As an example, I'm very much interested in compilers, virtual machines, programming language design and such, so have been plying around with these in various languages over the years. I wrote my first toy language in VB; wrote a few interpreters, virtual machines and simple parsers in C++ and some parsers, interpreters and assembly code generators in Python; for uni, I implemented a parser and code generator (instruction selection, using maximal munch) in JavaCC - recently I wrote an assembly code generator in Clojure. It took me a weekend and it surpassed the power of anything[1] I'd written in VB, C++, Python or Java. Its flexible and can be easily be extended, its pretty smart and can do some basic optimisation (caching data in registers, function inlining...). I tried writing something similar in Python before, but gave up.
My point is that I have much much deeper knowledge of C++, Python and Java, yet I was able to build something MORE complex and extensible in Clojure, in less time, even though I'd only been using it for a few months.
Yes, my previous experience played some part in this, but I attribute most of it to the flexibility and power Lisp provides me through easy to use and powerful abstractions, flexible and convenient syntax and interactive development.
[1] Of the same scale - any interpreter, code generator, compiler etc. I've written much more powerful systems, of course, but they are huge in comparison and took a lot lot longer than a weekend to write.
I would say that he has an enormous amount of experience with several languages. He lists on his webpage: "Programming languages used over the past 39-years include C/C++, Common Lisp/CLOS/MOP (and other Lisp varients), ALGOL, FORTRAN, PL/1, SNOBOL, numerous assembly languages, and others."
So, he probably could have written it in C++ (or SNOBOL ;-) ), but he chose to do it in Lisp.
No, he could not have (short of bootstraping a Lisp system from C++). I, for one, have much more C++ experience than Ocaml experience. I am far more efficient with Ocaml: typically, my Ocaml programs are 3 to 5 times shorter, takes half the time to program, and almost no time to debug.
I have 10 years of C++ experience, and 1 year of part-time Python experience. I needed to create a simple graph algorithm to find maximum Hamiltonian paths. It took me 1hr to get it running in Python and 6 hours to translate it to C++.
It is amazing how much Python (and other high level languages such as lisp, Ocaml) allow you to focus on the problem you are solving not the irrelevant details of the solution.
All things being equal, lisp was in use much more back in the 70s and 80s than it was in the 90s or 00s. It makes sense for the majority of people who would think to use lisp to be a bit grayer.
Fortunately, since 2000, a new generation of young hackers has shown increased passion for Lisp.
Enthusiasm, yes, but where is the output? Usually, when I read about a Lisp success story, there's a paragraph:
... and we could not have done this without A, B, C ... by Edi Weitz.
Lisp needs more people like him who actually release software instead of blogging about its virtues.
But you have seen the article that we are talking about? It is just such an output where the software is written in Lisp and the result is sold commercially.
People have asked Jack to explain why he used Lisp for an embedded application. You can read the result in the linked article.
The guy who has posted the story also has written a bunch of Lisp based internet applications. If you look among the hardcore Common Lisp users, quite a lot are writing software. The once that are blogging superficial stuff not so much. Read the stuff on planet.lisp.org - that's usually quite useful.
Why do you assume people don't release software in Common Lisp?
When you buy a piece of no one writes on the packaging "we did this in C++!" or "We did this in Java!" Why would you expect a big "Common Lisp!" sticker on the packaging? The deliverable is machine code, not source code. The customer doesn't care what the source is written in.
Because people don't write C++ or Java or Python "success stories". When someone ships an app in Lisp, they blog about it and it gets posted here. If Lisp were widely used, it would not be newsworthy.
I don't think you hear much about the 'real' Lisp apps. I have for example never seen anything about Amazon's use of Lisp. But they use it. Did you ever read something about the applications that schedule airport operations? Or did you ever hear about SISCOG's software ( http://www.siscog.com/ ) who develop most of their software in Lisp? Their products are used to schedule the crews of public transport systems (trains) in major cities and states.
Ever read blog posts about those?