Most of the complaints in this article boil down to "macros are too powerful." I think this is the key part of the argument:
"A smart programmer is not necessarily an empathetic language designer; they are occupations that require different skillsets. Giving any programmer on your team the ability to arbitrarily extend the compiler can lead to a bevy of strange syntax and hard-to-debug idiosyncrasies."
There are at least 2 counter-arguments to this:
1.) in the simplest case, just restrict the use of macros. A team can easily adapt the rule that only the most experienced engineer on the team is allowed to write or approve macros. (And in my experience, the need for macros is fairly rare. I think my ratio is something like 100 or 200 normal functions for every macro that I write.)
2.) macros allow all kinds of interesting type checking, and data structure validation, and therefore they make it surprisingly easy to validate data and types as your data and/or vars get passed around your system. Consider all of these very interesting tools you can use in the world of Clojure:
Prismatic Schema which allows validation that a data structure matches a schema of (possibly nested) types:
In short, there are an abundance of mechanisms available with Clojure which help facilitate the enforcement of any kind of schema or contract, and some of these tools are enabled (and their syntax is made clean) thanks to macros.
In short: macros can be used for evil, but they can also be used for good. They are very powerful, so everyone should be judicious about their use, but there is no reason to argue that macros render a Lisp unfit for programming in the large.
Having said all that, I'll remind everyone that the ultimate counter-argument is offered by Paul Graham, in his essay "Beating the averages":
The quote you pulled out boils down to something even simpler than that: People who are good at programming aren't necessarily good at designing API's.
I think there's a kernel of a valid point underneath. A macro necessarily has a larger (potential) interface surface than a function, because there's less you can take for granted about how it interacts with the rest of your code. And I agree, API design is a specialized skill that requires quite a bit of thought.
But I agree with your conclusion more than his: This is to take care with macros, not a reason to shun them altogether.
Precisely. The problem with the lisps of old was entirely cultural. People would go do crazy wild things and then not bother to interoperate with the rest of the world. Meanwhile, the Clojure community has lots of experimentation, but ultimately produces a large number of stable, quality, reusable libraries.
Large scale C++ teams often require approval for operator overloading or "dangerous" features. Can easily do the same for macros. Moreover, we now have distributed version control and can utilize lieutenant workflows, so we can dispatch with this silly "commit bit" notion that means bad code sneaks in past domain experts with ease.
Lisp has been already in times when the technology you are using today was still under invention. Lisp existed before Smalltalk, C, C++, Java, ... thus often technology was developed in an unstable surrounding where inventions are just being made. Lisp also had to keep track of the changing IT landscape. During the 70s people were using DEC PDP computers.
Thus you find evidence for everything.
There are well-documented stable, nicely reusable, code bases in Lisp.
Clojure is most of the time married to a small eco-system: Java/JVM.
Lisp has seen and supported many more eco-systems and will see even more in the future.
As a language geek, I tend to hunt the Internet for old papers and manuals related to OS and languages.
The actual mainstream situation could be so different if the Xerox PARC research in terms of programming languages and OS besides the GUI, had become mainstream instead of the AT&T ones.
The Interlisp, Smalltalk and Mesa systems were great computing platforms compared with what UNIX offered.
I tend to believe that with time, good genes that were previously trimmed for very pragmatic reasons, will reappear in more favorable contexts and spread.
It might be, but I imagine having an OS where Lisp is the systems programming language is way different from a system where it is just another language.
Yeah, but that does not make it 'better' or more useful.
Lisp-based operating systems are no longer used, because they were quite complex (coming out of a research environment) and provided LESS functionality in some crucial areas (for example they were not multi-user - they were one-person, one machine, one 'world'). Additionally they were expensive.
The Xerox Lisp and Xerox Smalltalk workstations one could buy were severely underpowered (RAM, speed, ...).
I don't think you can credibly describe Java/JVM as a small ecosystem.... there's nothing else remotely close in terms of the combination of runtime platform capabilities and the number of available open source libraries.
Enabling macros is the sole justification for the homoiconic syntax, which is often cited as the most offputting feature of lisps for uptake by large programmer teams. If you're going to have two "editions" of Clojure, one with the full feature set for language designers, and the other a more restricted sans-defmacro one for more general programmer use, then why not give that restricted one a more friendly syntax as well, or even just get them to use Java, Python, or whatever.
1) I genuinely prefer variadic prefix notation, even in the absence of homoiconicity.
2) Restricted does not mean "banned". It means that they need to be justified and subject to expert scrutiny.
3) Even if you never write a macro of your own, you're a beneficiary of the syntactic sugar and can leverage macroexpand to demystify otherwise opaque language constructs.
Pet peeve, sorry, but it irritates me that Clojure people act like this guilty-until-proven-innocent policy about macros is somehow original. It has been standard advice for decades. Chapter 8 of On Lisp (1994) is called "When to Use Macros" and its first section is "When Nothing Else Will Do":
By default we should use functions: it is inelegant to use a macro where a function would do. We should use macros only when they bring us some specific advantage.
I understand from a language marketing point of view why someone might say, "Oh, those other Lisps made wild and crazy use of macros. It was really bad! But we are enlightened and have restricted them." But it's a bogus way of playing to a bogus criticism. It would be better to just say that there's a tradition of how to use macros correctly.
You paint "Clojure people" - I'm one - with a pretty broad brush. If one of us has said something you disagree with, then please cite it along with your rebuttal.
It is unlikely that you have an informed opinion of "Clojure people" on the basis of a few posts that give you heartburn.
Macros have its major purpose - they are means of creating advanced DSLs. Simple DSLs could be created just out of high-order procedures and list structure, but they will be bounded by the general evaluation rule, that all the arguments would be evaluated before actual procedure application, which is not always what we want.
Macros is the way to add new special forms to a Lisp or a DSL embedded in it. This is why macros are there.
To avoud macros is to restrict oneself from designing a program as layers upon layers of DSLs which is the most powerful paradigm. Just look at that Rtml DSL of ViaWeb system.
I agree wholeheartedly, and would add:
The usage of macros is something that can (and should) be avoided in most cases, but in order for you to learn when one should or should not write macros, you should write programs large (or complex) enough that you would need to develop a domain-specific language for the problem at hand. The creation and use of small utility functions which reflect your growing understanding of the problem will remove some of the need to write macros, until you need to perform syntactic transformations. (Note that this is the case where writing a syntactic transformation is the simplest and fastest solution to whatever problem you're working on at the time). You usually don't know when this will happen until you need to do so, but it seems to be the safer option to use a Lisp, which allows you to do that very quickly.
I mostly agree but fwiw, the usual counterargument on #2 is that those should be language facilities instead, which allows them to be carefully designed and then taken advantage of by the compiler. For example, in Racket (formerly PLT Scheme) you have: http://docs.racket-lang.org/guide/contracts.html
This is a terrible example for your argument, because contracts in Racket are entirely implemented as a library, thanks to the power of -- you guessed it -- macros. In fact, just about everything in Racket: the class system, the generic function system, the unit system, the type system, the serialized continuations, the pattern matcher, keyword arguments -- all of those are implemented with macros.
The problem with this argument is, that you will never have the amount of independned development on every feature. There are usually only a relativly small amount of people at the core of a langague.
If I want a new feature in the language is very hard to get it in and once its in its in. With macros diffrent people can do there own thing and indepently develop it.
Look for example at core.match, in every other language something like it would have been a new language feature. Clojure now has a state of the art pattern matcher without Rich or anybody doing anything.
I'm not sure how well that specific example works. Are contracts built in to the language on a fundamental level, or are they just part of the standard library? Given that they're not included in racket/base, and given how flexible racket's core is, I would guess the latter, but I'm not sure.
"Are contracts built in to the language on a fundamental level, or are they just part of the standard library? "
One of the points of Lisp is that that difference doesn't matter.
But besides that, in this case it couldn't possibly matter. You write contracts for your functions. If they're violated at runtime you'll get a clear error that stops execution to contain damage, and assigns blame to the contract violator. At what point in that process does it matter whether contracts are "built into the language"?
To be clear, I'm not suggesting it's an important question per se.
_delerium presented an argument that the things people do with macros should be implemented as language facilities instead. Ve used racket contracts as an example.
I was saying that if racket contracts are not implemented as language facilities, it's not a very good example.
Comments
Most of the complaints in this article boil down to "macros are too powerful." I think this is the key part of the argument:
"A smart programmer is not necessarily an empathetic language designer; they are occupations that require different skillsets. Giving any programmer on your team the ability to arbitrarily extend the compiler can lead to a bevy of strange syntax and hard-to-debug idiosyncrasies."
There are at least 2 counter-arguments to this:
1.) in the simplest case, just restrict the use of macros. A team can easily adapt the rule that only the most experienced engineer on the team is allowed to write or approve macros. (And in my experience, the need for macros is fairly rare. I think my ratio is something like 100 or 200 normal functions for every macro that I write.)
2.) macros allow all kinds of interesting type checking, and data structure validation, and therefore they make it surprisingly easy to validate data and types as your data and/or vars get passed around your system. Consider all of these very interesting tools you can use in the world of Clojure:
Prismatic Schema which allows validation that a data structure matches a schema of (possibly nested) types:
https://github.com/prismatic/schema
and this now offers coercion, which makes this fantastic for importing JSON from other sub-systems or outside vendors:
http://blog.getprismatic.com/blog/2014/1/4/schema-020-back-w...
(I assume you could easily validate before giving data to Liberator to export your data while conforming to your schema: http://clojure-liberator.github.io/liberator/ )
There is work being done on an optional type system:
https://github.com/clojure/core.typed
Much effort has been made to make contract programming easy in Clojure:
https://github.com/clojure/core.contracts
But also I find the built-in syntax for writing pre and post assertions is clean and easy to use:
http://blog.fogus.me/2009/12/21/clojures-pre-and-post/
In short, there are an abundance of mechanisms available with Clojure which help facilitate the enforcement of any kind of schema or contract, and some of these tools are enabled (and their syntax is made clean) thanks to macros.
In short: macros can be used for evil, but they can also be used for good. They are very powerful, so everyone should be judicious about their use, but there is no reason to argue that macros render a Lisp unfit for programming in the large.
Having said all that, I'll remind everyone that the ultimate counter-argument is offered by Paul Graham, in his essay "Beating the averages":
http://www.paulgraham.com/avg.html
If that essay does not convince you of the value of macros/lisp, then nothing will.
The quote you pulled out boils down to something even simpler than that: People who are good at programming aren't necessarily good at designing API's.
I think there's a kernel of a valid point underneath. A macro necessarily has a larger (potential) interface surface than a function, because there's less you can take for granted about how it interacts with the rest of your code. And I agree, API design is a specialized skill that requires quite a bit of thought.
But I agree with your conclusion more than his: This is to take care with macros, not a reason to shun them altogether.
Precisely. The problem with the lisps of old was entirely cultural. People would go do crazy wild things and then not bother to interoperate with the rest of the world. Meanwhile, the Clojure community has lots of experimentation, but ultimately produces a large number of stable, quality, reusable libraries.
Large scale C++ teams often require approval for operator overloading or "dangerous" features. Can easily do the same for macros. Moreover, we now have distributed version control and can utilize lieutenant workflows, so we can dispatch with this silly "commit bit" notion that means bad code sneaks in past domain experts with ease.
Lisp has had widely different uses:
* a teaching language
* a research tool
* an application programming language
Lisp has been already in times when the technology you are using today was still under invention. Lisp existed before Smalltalk, C, C++, Java, ... thus often technology was developed in an unstable surrounding where inventions are just being made. Lisp also had to keep track of the changing IT landscape. During the 70s people were using DEC PDP computers.
Thus you find evidence for everything.
There are well-documented stable, nicely reusable, code bases in Lisp.
Clojure is most of the time married to a small eco-system: Java/JVM.
Lisp has seen and supported many more eco-systems and will see even more in the future.
As a language geek, I tend to hunt the Internet for old papers and manuals related to OS and languages.
The actual mainstream situation could be so different if the Xerox PARC research in terms of programming languages and OS besides the GUI, had become mainstream instead of the AT&T ones.
The Interlisp, Smalltalk and Mesa systems were great computing platforms compared with what UNIX offered.
I tend to believe that with time, good genes that were previously trimmed for very pragmatic reasons, will reappear in more favorable contexts and spread.
Still Lisp was on Unix on day two and was always a very popular platform for Lisp developers.
You could even get Lisp Machines from TI with embedded Unix and Lisp Machines from TI and Symbolics which were embedded in Unix.
Several Lisp companies made their entire business from Unix: Lucid, early Franz, early Harlequin/LispWorks, ...
It might be, but I imagine having an OS where Lisp is the systems programming language is way different from a system where it is just another language.
Yeah, but that does not make it 'better' or more useful.
Lisp-based operating systems are no longer used, because they were quite complex (coming out of a research environment) and provided LESS functionality in some crucial areas (for example they were not multi-user - they were one-person, one machine, one 'world'). Additionally they were expensive.
The Xerox Lisp and Xerox Smalltalk workstations one could buy were severely underpowered (RAM, speed, ...).
I don't think you can credibly describe Java/JVM as a small ecosystem.... there's nothing else remotely close in terms of the combination of runtime platform capabilities and the number of available open source libraries.
Enabling macros is the sole justification for the homoiconic syntax, which is often cited as the most offputting feature of lisps for uptake by large programmer teams. If you're going to have two "editions" of Clojure, one with the full feature set for language designers, and the other a more restricted sans-defmacro one for more general programmer use, then why not give that restricted one a more friendly syntax as well, or even just get them to use Java, Python, or whatever.
1) I genuinely prefer variadic prefix notation, even in the absence of homoiconicity.
2) Restricted does not mean "banned". It means that they need to be justified and subject to expert scrutiny.
3) Even if you never write a macro of your own, you're a beneficiary of the syntactic sugar and can leverage macroexpand to demystify otherwise opaque language constructs.
Pet peeve, sorry, but it irritates me that Clojure people act like this guilty-until-proven-innocent policy about macros is somehow original. It has been standard advice for decades. Chapter 8 of On Lisp (1994) is called "When to Use Macros" and its first section is "When Nothing Else Will Do":
By default we should use functions: it is inelegant to use a macro where a function would do. We should use macros only when they bring us some specific advantage.
I understand from a language marketing point of view why someone might say, "Oh, those other Lisps made wild and crazy use of macros. It was really bad! But we are enlightened and have restricted them." But it's a bogus way of playing to a bogus criticism. It would be better to just say that there's a tradition of how to use macros correctly.
Please name a language you use so that whenever you and I disagree I'll be able to cite the "Blub people".
Not sure what your point is? I'd be happy to be wrong here.
You paint "Clojure people" - I'm one - with a pretty broad brush. If one of us has said something you disagree with, then please cite it along with your rebuttal.
It is unlikely that you have an informed opinion of "Clojure people" on the basis of a few posts that give you heartburn.
Uh, I think maybe I'd better just quit while I'm behind.
Macros have its major purpose - they are means of creating advanced DSLs. Simple DSLs could be created just out of high-order procedures and list structure, but they will be bounded by the general evaluation rule, that all the arguments would be evaluated before actual procedure application, which is not always what we want.
Macros is the way to add new special forms to a Lisp or a DSL embedded in it. This is why macros are there.
To avoud macros is to restrict oneself from designing a program as layers upon layers of DSLs which is the most powerful paradigm. Just look at that Rtml DSL of ViaWeb system.
I agree wholeheartedly, and would add: The usage of macros is something that can (and should) be avoided in most cases, but in order for you to learn when one should or should not write macros, you should write programs large (or complex) enough that you would need to develop a domain-specific language for the problem at hand. The creation and use of small utility functions which reflect your growing understanding of the problem will remove some of the need to write macros, until you need to perform syntactic transformations. (Note that this is the case where writing a syntactic transformation is the simplest and fastest solution to whatever problem you're working on at the time). You usually don't know when this will happen until you need to do so, but it seems to be the safer option to use a Lisp, which allows you to do that very quickly.
I mostly agree but fwiw, the usual counterargument on #2 is that those should be language facilities instead, which allows them to be carefully designed and then taken advantage of by the compiler. For example, in Racket (formerly PLT Scheme) you have: http://docs.racket-lang.org/guide/contracts.html
This is a terrible example for your argument, because contracts in Racket are entirely implemented as a library, thanks to the power of -- you guessed it -- macros. In fact, just about everything in Racket: the class system, the generic function system, the unit system, the type system, the serialized continuations, the pattern matcher, keyword arguments -- all of those are implemented with macros.
The problem with this argument is, that you will never have the amount of independned development on every feature. There are usually only a relativly small amount of people at the core of a langague.
If I want a new feature in the language is very hard to get it in and once its in its in. With macros diffrent people can do there own thing and indepently develop it.
Look for example at core.match, in every other language something like it would have been a new language feature. Clojure now has a state of the art pattern matcher without Rich or anybody doing anything.
I'm not sure how well that specific example works. Are contracts built in to the language on a fundamental level, or are they just part of the standard library? Given that they're not included in racket/base, and given how flexible racket's core is, I would guess the latter, but I'm not sure.
"Are contracts built in to the language on a fundamental level, or are they just part of the standard library? "
One of the points of Lisp is that that difference doesn't matter.
But besides that, in this case it couldn't possibly matter. You write contracts for your functions. If they're violated at runtime you'll get a clear error that stops execution to contain damage, and assigns blame to the contract violator. At what point in that process does it matter whether contracts are "built into the language"?
To be clear, I'm not suggesting it's an important question per se.
_delerium presented an argument that the things people do with macros should be implemented as language facilities instead. Ve used racket contracts as an example.
I was saying that if racket contracts are not implemented as language facilities, it's not a very good example.