If I was manager wanting to decide on a substantial technology change like Clojure with a presentation like that I'd have to answer "no". It doesn't answer some of the most important questions:
1) What level of support is there for the Clojure toolchain? Is this likely to be around in 5 years from now?
2) We are going to need to hire programmers in the future. Are there people with Clojure skills (or those that we can cross-train) around? In our geographic region? At the price point we can afford? In 5 years from now is it likely that this is going to be a marginalized technology that we have trouble finding skills for?
These are the more important questions, not how succinct the language is. I've had first-hand experience of how technology choices affect companies and it can be very painful down the road.
(Note: this isn't intended as a dig at Clojure, you could replace it with Scala, or any other up-and-coming language choice.)
While I wholly sympathize with your rather pragmatic pov, technology is not a staple industry like agriculture or the diary business or the furniture industry. It is more like the fashion industry. You have to be a bit of a visionary to succeed in this trade.
>Is this likely to be around in 5 years from now?
In 1997 when I graduated, career services told me to erase Java from my resume and replace it with "Borland C++ OWL 2.0" so that I could land a job. (http://en.wikipedia.org/wiki/Object_Windows_Library) At that time, Java = dancing duke on netscape navigator. Right now, I'd give a million dollars to find another OWL 2.0 developer ( ok not a million, maybe $10 )
>5 years from now is it likely that this is going to be a marginalized technology
5 years == light years in tech. Nobody can predict that far out.
>it can be very painful down the road.
If you future-proof yourself by only using the most stable, well supported toolchain, it can be much more painful down the road when you are stuck with some legacy platform nobody is willing to code in because everybody has moved on to the next big thing.
>you could replace it with Scala, or any other up-and-coming language
There should be a statute of limitations on when you can stop using "up-and-coming" w.r.t a language :) Seriously,Scala has been around in some shape/form since 2003. We've had stable releases since 2008. At this point it is no longer "up-and-coming".
I surely can predict that in 5 "light" years C++, even C, will not become obsolete. I know of a very innovative company which still uses almost exclusively Visual C++ 2008 (they do some heavy image processing and OCR).
> We are going to need to hire programmers in the future. Are there people with Clojure skills (or those that we can cross-train) around? In our geographic region? At the price point we can afford?
I think this common perspective is a little misguided. Programmers retrain fairly easily, inexpensive recent graduates doubly so. With JVM languages, it's particularly true, because the overall environment is so large compared to the language, and all that comes over for free.
I don't really understand the business world's preoccupation with 5 years of experience. My experience has been that some programmers "get it" right out of school, others never get it, some continue to improve into their 60s and beyond. Experience levels correlate strongly with desired starting salary, but not with skill or ability to learn.
> In 5 years from now is it likely that this is going to be a marginalized technology that we have trouble finding skills for?
Lisp was a highly marginalized technology for a few decades there. No doubt you could scour résumés at Monster.com all day and not find anyone who had bothered to put it down. But nobody who actually tried to hire Common Lisp programmers had much difficulty finding them or paying them, because of the prestige. Very few languages seem to enter an actively-hated state (Cobol and Mumps being obvious exceptions); the rest enter disuse and we programmers get all misty-eyed for them.
In short, this is one of those "obvious" business fears that turns out to be completely irrational.
On the flip side, where I work, we use Java 6, JSF 2 and Hibernate: industry standard stuff. Nevertheless, we've been able to fill only one of three open positions in the last three years, after multiple month searches. The size of the labor pool is usually not the limiting factor: in our case, we are in the middle of nowhere and offer below industry standard compensation (though it's very high for the region). Where you are, what you pay, what the benefits are like, what you work on; these kinds of things are going to have a much more dramatic effect on your ability to find developers than simply choosing an off-beat language. Plenty of recent graduates would rather work for less money on interesting problems with desirable tools.
> I've had first-hand experience of how technology choices affect companies and it can be very painful down the road.
I've had the same experience. A couple of years ago we decided to implement a rather large part of a project in JRuby (the other part was written in Java), because it seemed to be the right tool for the job: a scripting language that can interoperate with java libraries out of the box.
In retrospect it was a bad decision, because the interpreter error messages were (are?) cryptic (at least compared to what you get with Java) and IDE integration was really, really bad. Cross-language re-factoring was broken and we ended up unit testing almost every line of JRuby code to make sure we didn't introduce regressions when making changes to running systems.
There's a part in Peopleware (http://www.amazon.com/Peopleware-Productive-Projects-Teams-2...) where the authors describe that the choice of programming language has almost no impact on the success of a project. So why add additional risk when there's no proven benefit?
Edit: Also, people always argue that they can get things done much faster in "esoteric" languages, but forget about the time spent on maintenance, bug fixing, explaining code to new colleagues, ...
Just a quick note: from personal experience, I've been a lot more productive in Clojure than in Java, and I have written a lot of Java code. Things like the REPL, STM or FP speed you up a lot, I would say by a factor of at least 2 - 3, maybe more. I have no data to back it up but this is my perception.
So hiring one programmer instead of 3 and paying him 2x more is already good for you.
When I switched from C to Java, I experienced a considerable bump in development speed. Just the collections framework was a godsend, not to mention things like GC etc.
Quite literally I would write things in days instead of weeks. Same thing goes for Java -> Clojure. And just to be clear, it is not my favorite language.
Funny story. I programmed briefly in C and went to Java. I had almost five years of experience there. Due to circumstances, my sister now needs some help coding some stuff in C (note these aren't same types of problems I had). And I noticed that I am now so much more productive in C compared to what I was before. Hell I make less mistakes than in Java (granted problems are a bit easier).
My point is - are you sure you just aren't better programmer overall and attributing that (somewhat underservedly because GC and a huge choice of collections) to Java/Clojure/<insert language>?
Partly, sure. If I had to write some C again I would surely try to do it as functionally as possible, and with the added experience I have gained in the last years I would now probably be a better overall C programmer. But had I sticked with C and not moved on to other languages I probably wouldn't be much better now than I was before.
I'd hazard a guess that this had less to do with java and more to do with your lack of c experience. collections in c is a solved problem, there are numerous libraries (glib, for example) out there that are all similar to Java's collections.
C++ has stl with many of the same collections, and if you want to use auto_ptr, you don't even need to remember to call delete.
I am not saying there are not productivity gains to be had from using java, but as in this case, it typically comes from not having the relevant experience elsewhere.
Automatic memory management, instantaneous compilation, no portability issues, (easier) polymorphism. Maybe weeks -> days was an overexaggeration when it comes to C -> Java but I definitely experienced a big productivity gain, and it wasn't because of a lack of C experience.
From a quick glance at the docs glib looks good but it is still too low-level for normal app development (for example, the size of a hashing table created by hcreate() is fixed; so you have to create & copy & destroy a hash table if you want to add more elements than it can hold at a time) and it should be, that's the whole point.
However, it's also widely held that individual programmer ability and the cultures of certain groups can have even larger affects on productivity.
Switching languages is hard. Managing groups is hard. Dealing with the rest of a large organization is hard. If a group can do fine without switching languages, why would any manager be in a hurry to do it?
But you're depending on 1 programmers instead of spreading your risk into 3 different developers.
Lots of angles to consider not just in terms of monetary. Human's behaviour/attitude/mindset is a very large unknown despite however hard we tried to make a "human" as static as a "resource".
Sure, but a 3x faster programmer will still be more efficient than 3 other programmers, because there is no communication overhead and opportunities for misunderstandings.
I would rather hire somebody really good for $6k / month than 3 people for $2k each.
Maybe you don't want to replace three java programmers with one clojure programmer, but it might not be so bad to replace fifteen java programmers with five clojure programmers.
I would definitely agree with all the advantages of having less (less people to manage, less communication, less lines of code) provided that you can find the ideal situation.
But alas, finding the ideal situation is like winning the NBA Championship trophy: you need all the stars to line up including "Lucky" star.
Clojure and Java aren't really direct competitors. The appropriate comparison is to Python, Ruby, or Perl. I strongly doubt there would be a 2-3x difference in that case.
Personally speaking, even though I really wanted to love Clojure, I was forced to conclude that Python is more productive.
Well, it depends. For example, look at core.logic and show me something comparable in Perl or Python. The ability to just code up a DSL in no time, with the same syntax as the rest of the language is a really powerful tool, and I've yet to see a language that comes even close to Lisp's metaprogramming support.
Python has logic programming (Pyke). But, in principle, yes: you can't add new syntax to non-homoiconic languages. You can use macros (even C has macros), but they are more limited in a nonhomoiconic language.
In my experience, though, metaprogramming is of limited use as anything other than syntactic sugar. And it often leads to very confusing code.
Other language features are far more fundamental and important: libraries, good integration with the shell, coherent error messages (!), readability, tooling, deployment, etc. As a language Clojure is beautiful, but as an ecosystem it has very far to go.
I'm not aware of many non-Lisp non-Prolog logic programming libraries like core.logic. core.logic can trivially operate on the terms of language which it is defined in. Meaning core.logic itself can be used for metaprogramming.
In my experience, metaprogramming is a wonderfully powerful utility for controlling complexity and confusion.
As far as the ecosystem - I used Python for a couple years, it's good. But the Clojure ecosystem is doing pretty darn well itself these days.
Theses are really common questions to ask and we already know the responses.
But are you asking the right questions?
Your hypothesis is that anybody suffiently trained is enough and it is easy to scale by having more people.
This work well for contracting companies that effectivelly sell people and not really care about the software. Their model is totally scalable: they revenue is mostly dependant of the number of people billed. And too much productivy is more a nuisance. They just need not to be worse than other contracting companies. So do the same as everybody else, be sure to have low operationnal costs and you are fine.
But from the software side, this is the opposite. Software don't scale. The bigger the code base, the more it cost to maintain. Too many people working on it and too much turnover and your team start to spend more time in meetings, politics or helping newcomers than getting things done.
As a manager, while this is far more satisfactory to manage bigger teams, your role is to empower your team so it can do more with less. And any tool that can give you an edge is a huge asset. If a better programming language mean more productivity, use it. If a better SCM can help, use it! If faster computer ensure that your devs stay in the zone, buy them.
That the point here. You can do more with a good team of 10 people than with an average team of 50 people (where 10 peoples will be consumed just to manage others).
Comments
If I was manager wanting to decide on a substantial technology change like Clojure with a presentation like that I'd have to answer "no". It doesn't answer some of the most important questions:
1) What level of support is there for the Clojure toolchain? Is this likely to be around in 5 years from now?
2) We are going to need to hire programmers in the future. Are there people with Clojure skills (or those that we can cross-train) around? In our geographic region? At the price point we can afford? In 5 years from now is it likely that this is going to be a marginalized technology that we have trouble finding skills for?
These are the more important questions, not how succinct the language is. I've had first-hand experience of how technology choices affect companies and it can be very painful down the road.
(Note: this isn't intended as a dig at Clojure, you could replace it with Scala, or any other up-and-coming language choice.)
While I wholly sympathize with your rather pragmatic pov, technology is not a staple industry like agriculture or the diary business or the furniture industry. It is more like the fashion industry. You have to be a bit of a visionary to succeed in this trade.
>Is this likely to be around in 5 years from now?
In 1997 when I graduated, career services told me to erase Java from my resume and replace it with "Borland C++ OWL 2.0" so that I could land a job. (http://en.wikipedia.org/wiki/Object_Windows_Library) At that time, Java = dancing duke on netscape navigator. Right now, I'd give a million dollars to find another OWL 2.0 developer ( ok not a million, maybe $10 )
>5 years from now is it likely that this is going to be a marginalized technology
5 years == light years in tech. Nobody can predict that far out.
>it can be very painful down the road.
If you future-proof yourself by only using the most stable, well supported toolchain, it can be much more painful down the road when you are stuck with some legacy platform nobody is willing to code in because everybody has moved on to the next big thing.
>you could replace it with Scala, or any other up-and-coming language
There should be a statute of limitations on when you can stop using "up-and-coming" w.r.t a language :) Seriously,Scala has been around in some shape/form since 2003. We've had stable releases since 2008. At this point it is no longer "up-and-coming".
I agree with you overall but the sad state of Scala's build tools make it still "up-and-coming" in my eyes.
I surely can predict that in 5 "light" years C++, even C, will not become obsolete. I know of a very innovative company which still uses almost exclusively Visual C++ 2008 (they do some heavy image processing and OCR).
Pedantic point: Light years are a unit of distance, not time.
http://en.wikipedia.org/wiki/Lightyear
So it's further ahead and you can't see what will happen there. Sounds like a perfectly good use of the word, given the context :).
> We are going to need to hire programmers in the future. Are there people with Clojure skills (or those that we can cross-train) around? In our geographic region? At the price point we can afford?
I think this common perspective is a little misguided. Programmers retrain fairly easily, inexpensive recent graduates doubly so. With JVM languages, it's particularly true, because the overall environment is so large compared to the language, and all that comes over for free.
I don't really understand the business world's preoccupation with 5 years of experience. My experience has been that some programmers "get it" right out of school, others never get it, some continue to improve into their 60s and beyond. Experience levels correlate strongly with desired starting salary, but not with skill or ability to learn.
> In 5 years from now is it likely that this is going to be a marginalized technology that we have trouble finding skills for?
Lisp was a highly marginalized technology for a few decades there. No doubt you could scour résumés at Monster.com all day and not find anyone who had bothered to put it down. But nobody who actually tried to hire Common Lisp programmers had much difficulty finding them or paying them, because of the prestige. Very few languages seem to enter an actively-hated state (Cobol and Mumps being obvious exceptions); the rest enter disuse and we programmers get all misty-eyed for them.
In short, this is one of those "obvious" business fears that turns out to be completely irrational.
On the flip side, where I work, we use Java 6, JSF 2 and Hibernate: industry standard stuff. Nevertheless, we've been able to fill only one of three open positions in the last three years, after multiple month searches. The size of the labor pool is usually not the limiting factor: in our case, we are in the middle of nowhere and offer below industry standard compensation (though it's very high for the region). Where you are, what you pay, what the benefits are like, what you work on; these kinds of things are going to have a much more dramatic effect on your ability to find developers than simply choosing an off-beat language. Plenty of recent graduates would rather work for less money on interesting problems with desirable tools.
> I've had first-hand experience of how technology choices affect companies and it can be very painful down the road.
I've had the same experience. A couple of years ago we decided to implement a rather large part of a project in JRuby (the other part was written in Java), because it seemed to be the right tool for the job: a scripting language that can interoperate with java libraries out of the box.
In retrospect it was a bad decision, because the interpreter error messages were (are?) cryptic (at least compared to what you get with Java) and IDE integration was really, really bad. Cross-language re-factoring was broken and we ended up unit testing almost every line of JRuby code to make sure we didn't introduce regressions when making changes to running systems.
There's a part in Peopleware (http://www.amazon.com/Peopleware-Productive-Projects-Teams-2...) where the authors describe that the choice of programming language has almost no impact on the success of a project. So why add additional risk when there's no proven benefit?
Edit: Also, people always argue that they can get things done much faster in "esoteric" languages, but forget about the time spent on maintenance, bug fixing, explaining code to new colleagues, ...
At the price point we can afford?
Just a quick note: from personal experience, I've been a lot more productive in Clojure than in Java, and I have written a lot of Java code. Things like the REPL, STM or FP speed you up a lot, I would say by a factor of at least 2 - 3, maybe more. I have no data to back it up but this is my perception.
So hiring one programmer instead of 3 and paying him 2x more is already good for you.
Everyone claims this about their favorite language. If I were a manager, I sure as hell wouldn't believe it.
Good you're not a manager then.
When I switched from C to Java, I experienced a considerable bump in development speed. Just the collections framework was a godsend, not to mention things like GC etc.
Quite literally I would write things in days instead of weeks. Same thing goes for Java -> Clojure. And just to be clear, it is not my favorite language.
neutronicus should read the Blug Paradox in pg's Beating the Averages (http://www.paulgraham.com/avg.html).
Funny story. I programmed briefly in C and went to Java. I had almost five years of experience there. Due to circumstances, my sister now needs some help coding some stuff in C (note these aren't same types of problems I had). And I noticed that I am now so much more productive in C compared to what I was before. Hell I make less mistakes than in Java (granted problems are a bit easier).
My point is - are you sure you just aren't better programmer overall and attributing that (somewhat underservedly because GC and a huge choice of collections) to Java/Clojure/<insert language>?
Partly, sure. If I had to write some C again I would surely try to do it as functionally as possible, and with the added experience I have gained in the last years I would now probably be a better overall C programmer. But had I sticked with C and not moved on to other languages I probably wouldn't be much better now than I was before.
I'd hazard a guess that this had less to do with java and more to do with your lack of c experience. collections in c is a solved problem, there are numerous libraries (glib, for example) out there that are all similar to Java's collections.
C++ has stl with many of the same collections, and if you want to use auto_ptr, you don't even need to remember to call delete.
I am not saying there are not productivity gains to be had from using java, but as in this case, it typically comes from not having the relevant experience elsewhere.
Automatic memory management, instantaneous compilation, no portability issues, (easier) polymorphism. Maybe weeks -> days was an overexaggeration when it comes to C -> Java but I definitely experienced a big productivity gain, and it wasn't because of a lack of C experience.
From a quick glance at the docs glib looks good but it is still too low-level for normal app development (for example, the size of a hashing table created by hcreate() is fixed; so you have to create & copy & destroy a hash table if you want to add more elements than it can hold at a time) and it should be, that's the whole point.
C++ is a whole other story, sure.
Everyone claims this about their favorite language. If I were a manager, I sure as hell wouldn't believe it.
Well, it's been known for awhile that languages and environments can have a 2X or 3X influence on programmer productivity.
http://www.qsm.com/resources/function-point-languages-table
However, it's also widely held that individual programmer ability and the cultures of certain groups can have even larger affects on productivity.
Switching languages is hard. Managing groups is hard. Dealing with the rest of a large organization is hard. If a group can do fine without switching languages, why would any manager be in a hurry to do it?
I've never heard anyone say this about C... If I've heard it said about C++, that was a long, long time ago.
But you're depending on 1 programmers instead of spreading your risk into 3 different developers.
Lots of angles to consider not just in terms of monetary. Human's behaviour/attitude/mindset is a very large unknown despite however hard we tried to make a "human" as static as a "resource".
Sure, but a 3x faster programmer will still be more efficient than 3 other programmers, because there is no communication overhead and opportunities for misunderstandings.
I would rather hire somebody really good for $6k / month than 3 people for $2k each.
Sure but people need to go on vacation, multiple projects need to be worked simultaneously, developers may leave their jobs, etc.
There are lots of good reasons to have multiple developers on a project other than simply throughput.
Maybe you don't want to replace three java programmers with one clojure programmer, but it might not be so bad to replace fifteen java programmers with five clojure programmers.
I would definitely agree with all the advantages of having less (less people to manage, less communication, less lines of code) provided that you can find the ideal situation.
But alas, finding the ideal situation is like winning the NBA Championship trophy: you need all the stars to line up including "Lucky" star.
These really good people are quite unstable (moving on or burning out), so the risk is higher.
Me too, unless ofcourse I only have $2k in which case I'll be forced to just higher the the later kind of programmer.
Clojure and Java aren't really direct competitors. The appropriate comparison is to Python, Ruby, or Perl. I strongly doubt there would be a 2-3x difference in that case.
Personally speaking, even though I really wanted to love Clojure, I was forced to conclude that Python is more productive.
Well, it depends. For example, look at core.logic and show me something comparable in Perl or Python. The ability to just code up a DSL in no time, with the same syntax as the rest of the language is a really powerful tool, and I've yet to see a language that comes even close to Lisp's metaprogramming support.
Python has logic programming (Pyke). But, in principle, yes: you can't add new syntax to non-homoiconic languages. You can use macros (even C has macros), but they are more limited in a nonhomoiconic language.
In my experience, though, metaprogramming is of limited use as anything other than syntactic sugar. And it often leads to very confusing code.
Other language features are far more fundamental and important: libraries, good integration with the shell, coherent error messages (!), readability, tooling, deployment, etc. As a language Clojure is beautiful, but as an ecosystem it has very far to go.
I'm not aware of many non-Lisp non-Prolog logic programming libraries like core.logic. core.logic can trivially operate on the terms of language which it is defined in. Meaning core.logic itself can be used for metaprogramming.
In my experience, metaprogramming is a wonderfully powerful utility for controlling complexity and confusion.
As far as the ecosystem - I used Python for a couple years, it's good. But the Clojure ecosystem is doing pretty darn well itself these days.
Theses are really common questions to ask and we already know the responses.
But are you asking the right questions?
Your hypothesis is that anybody suffiently trained is enough and it is easy to scale by having more people.
This work well for contracting companies that effectivelly sell people and not really care about the software. Their model is totally scalable: they revenue is mostly dependant of the number of people billed. And too much productivy is more a nuisance. They just need not to be worse than other contracting companies. So do the same as everybody else, be sure to have low operationnal costs and you are fine.
But from the software side, this is the opposite. Software don't scale. The bigger the code base, the more it cost to maintain. Too many people working on it and too much turnover and your team start to spend more time in meetings, politics or helping newcomers than getting things done.
As a manager, while this is far more satisfactory to manage bigger teams, your role is to empower your team so it can do more with less. And any tool that can give you an edge is a huge asset. If a better programming language mean more productivity, use it. If a better SCM can help, use it! If faster computer ensure that your devs stay in the zone, buy them.
That the point here. You can do more with a good team of 10 people than with an average team of 50 people (where 10 peoples will be consumed just to manage others).