Every time I see a post from Lisp fans about macros, I want to be amazed, but I always just walk away confused. I can tell there's something interesting in there, but the quote-unquote-quasiquote syntax is just so dense that my brain is incapable of comprehending it.
`(OK ,(+ 10 20) == 30) ; result: (OK 30 == 30) — a list of 4 atoms
In my toy Lisp I used $ instead of comma, which I found a tad "more readable" coming from today's JS, Perl, Bash world. (And makes quasiquote... "the money quote"? =)
Focus on the problem that is being solved, not the mechanism. Consider this classic trick:
handle = (exists(file) and open(file))
That uses the short-circuiting behaviour of && to only open a file if it exists. Not great practice to write it in that way IMO but whatever. You can't implement it as a function:
handle = and(exists(file), open(file))
def and(exists, file):
... No viable implementation in Python afaik ...
That doesn't work because functions arguments don't/can't implement short circuiting - both arguments are evaluated before our function is called (although we could make it work in this case by passing the file name in - the point is that the function signature has to change).
Macros however can implement something that has short circuiting behaviour. It is a mechanism for implementing new syntax - more powerful than functions but as a trade off more error prone. Quasiquoting and whatever falls out of that as implementation details once you've got all that as context.
I know it’s not the point of your example, but you should not check if a file exists right before opening it. It could get deleted in between exists(file) and open(file), so you still have to handle the case where opening fails with FileNotFound.
I find myself increasingly preferring the exists()-then-open() sequence to open()-and-catch-NotFound (not just in Python, but generally).
Yes, it is classically frowned upon, but:
1. It is typical that there are multiple checks that need to be done in addition to exists(). It is a given that conditionals are more flexible than exceptions (even in Python). As long as some checks are in conditionals, it makes the code more legible and easier to reason about when all preliminary checks are in the same fashion and in the same place.
2. You still need to handle the other possible sub-varieties of OSError and other exceptions that can arise due to, say, I/O being famously unreliable. If you implement that handling gracefully, you will have the FileNotFoundError covered for free. From end user experience perspective, since a file being deleted between exists() and immediate open() on a single system within a single thread is an out of ordinary one-in-a-thousand-years case, it is OK if it is handled slightly less gracefully than if it occurs during the preliminary checks.
You still have to handle the exceptional case, but also checking beforehand can lead to better error messages, and in situations involving multiple resources can avoid having to rollback a second resource in the non-racy case.
I think everybody gets that (basic understanding of macros); but that’s not the level of apparent sophistication TFA is about. I agree with the GP comment.
We'll need a mechanism to suppress evaluation (quote '), we want a mechanism to trigger evaluation in a quote (unquote ,) but because we aren't barbarians we don't allow unquoting inside a quote since that would be confusing, so unquote is only meaningful in a quasiquote (ie, a quote that allows unquoting ~) and then there is splice syntax (;) because that is a convenient operation. That is a mumbo-jumbo explanation and the real one involves some treatment of the read-eval-print loop, symbol resolution and whatnot I dunno about Janet; but it isn't that badly wrong.
It is annoying to use the first few times because macros demand an unusually precise understanding about how evaluation works, but that is to some extent the point of the article - working through some things that could be misunderstood about how to build an unevaluated blob of code. It'll look weird to people who use languages without macros because there isn't a reason to get into the weeds of evaluation ... unless writing new syntax with macros.
Work that we're used to offloading to syntax is instead carried by your brain.
Kinda like the challenge to eating salad is all the chewing. It's not just that the brain needs to parse a lot more language to express a simple concept; it's also that the fingers also need to type all that out.
Yeah this is kinda like... this is definitely an advanced topic within the topic of macro writing and not very intelligible if you haven't written simpler macros for a while already.
I wrote a book whose third chapter is a much "gentler" introduction to macros. I don't know if it's actually intelligible (see: the monad tutorial fallacy) but it presents them the way that I was first able to understand them -- explicitly starting without quasiquote and working up to it. Easier for me than getting lost in the notation. https://janet.guide/macros-and-metaprogramming/
Yep, they are a foreign idea in pretty much all languages, but they are super easy once you figure them out.
If anyone actually wants to get their hands dirty to learn about Lisp macros, I recommend picking a Lisp implementation like SBCL, GNU Guile, Emacs, Clojure, or Hylang depending on what kind of environment you're comfortable with. The key about each of the Lisp implementations I mentioned here is that they all support "Common Lisp style macros", which are the bare bones most obvious way to do macros in Lisp.
Then I recommend using your choice of Lisp to implement a language feature you use in another language. It doesn't matter if that language feature already exists in your choice of Lisp, you can still implement it yourself. For example, you can choose to implement C-style for loops or while loops, asynchronous coroutines like Go, pattern matching, lambdas, whatever. I actually implemented asnyc/await in IronScheme and pushed it upstream[0].
If you want to read more about Lisp macros, I have really enjoyed the book Let over Lambda. I have also heard a lot about On Lisp by pg, but I haven't read that myself yet. Also if you really want to dive off the deep end into the beauty of programming, I recommend SICP.
Comments
Every time I see a post from Lisp fans about macros, I want to be amazed, but I always just walk away confused. I can tell there's something interesting in there, but the quote-unquote-quasiquote syntax is just so dense that my brain is incapable of comprehending it.
Here's JS string interpolation:
`OK ${10+20} == 30` // result: "OK 30 == 30" — a JS string
Here's a quasiquote-unquote:
`(OK ,(+ 10 20) == 30) ; result: (OK 30 == 30) — a list of 4 atoms
In my toy Lisp I used $ instead of comma, which I found a tad "more readable" coming from today's JS, Perl, Bash world. (And makes quasiquote... "the money quote"? =)
Focus on the problem that is being solved, not the mechanism. Consider this classic trick:
That uses the short-circuiting behaviour of && to only open a file if it exists. Not great practice to write it in that way IMO but whatever. You can't implement it as a function: That doesn't work because functions arguments don't/can't implement short circuiting - both arguments are evaluated before our function is called (although we could make it work in this case by passing the file name in - the point is that the function signature has to change).Macros however can implement something that has short circuiting behaviour. It is a mechanism for implementing new syntax - more powerful than functions but as a trade off more error prone. Quasiquoting and whatever falls out of that as implementation details once you've got all that as context.
I know it’s not the point of your example, but you should not check if a file exists right before opening it. It could get deleted in between exists(file) and open(file), so you still have to handle the case where opening fails with FileNotFound.
I find myself increasingly preferring the exists()-then-open() sequence to open()-and-catch-NotFound (not just in Python, but generally).
Yes, it is classically frowned upon, but:
1. It is typical that there are multiple checks that need to be done in addition to exists(). It is a given that conditionals are more flexible than exceptions (even in Python). As long as some checks are in conditionals, it makes the code more legible and easier to reason about when all preliminary checks are in the same fashion and in the same place.
2. You still need to handle the other possible sub-varieties of OSError and other exceptions that can arise due to, say, I/O being famously unreliable. If you implement that handling gracefully, you will have the FileNotFoundError covered for free. From end user experience perspective, since a file being deleted between exists() and immediate open() on a single system within a single thread is an out of ordinary one-in-a-thousand-years case, it is OK if it is handled slightly less gracefully than if it occurs during the preliminary checks.
You still have to handle the exceptional case, but also checking beforehand can lead to better error messages, and in situations involving multiple resources can avoid having to rollback a second resource in the non-racy case.
I think everybody gets that (basic understanding of macros); but that’s not the level of apparent sophistication TFA is about. I agree with the GP comment.
Well then, extending my example:
We'll need a mechanism to suppress evaluation (quote '), we want a mechanism to trigger evaluation in a quote (unquote ,) but because we aren't barbarians we don't allow unquoting inside a quote since that would be confusing, so unquote is only meaningful in a quasiquote (ie, a quote that allows unquoting ~) and then there is splice syntax (;) because that is a convenient operation. That is a mumbo-jumbo explanation and the real one involves some treatment of the read-eval-print loop, symbol resolution and whatnot I dunno about Janet; but it isn't that badly wrong.
It is annoying to use the first few times because macros demand an unusually precise understanding about how evaluation works, but that is to some extent the point of the article - working through some things that could be misunderstood about how to build an unevaluated blob of code. It'll look weird to people who use languages without macros because there isn't a reason to get into the weeds of evaluation ... unless writing new syntax with macros.
Paul Graham's https://paulgraham.com/onlisp.html is a whole book about it that really helped it click for me.
The challenge with the syntax is that there is no syntax. Work that we're used to offloading to syntax is instead carried by your brain.
Kinda like the challenge to eating salad is all the chewing. It's not just that the brain needs to parse a lot more language to express a simple concept; it's also that the fingers also need to type all that out.
In neither case should it be a serious issue.
Yeah this is kinda like... this is definitely an advanced topic within the topic of macro writing and not very intelligible if you haven't written simpler macros for a while already.
I wrote a book whose third chapter is a much "gentler" introduction to macros. I don't know if it's actually intelligible (see: the monad tutorial fallacy) but it presents them the way that I was first able to understand them -- explicitly starting without quasiquote and working up to it. Easier for me than getting lost in the notation. https://janet.guide/macros-and-metaprogramming/
To get it you really need to learn enough lisp (not a lot) and try implementing a non-trivial macro.
Yep, they are a foreign idea in pretty much all languages, but they are super easy once you figure them out.
If anyone actually wants to get their hands dirty to learn about Lisp macros, I recommend picking a Lisp implementation like SBCL, GNU Guile, Emacs, Clojure, or Hylang depending on what kind of environment you're comfortable with. The key about each of the Lisp implementations I mentioned here is that they all support "Common Lisp style macros", which are the bare bones most obvious way to do macros in Lisp.
Then I recommend using your choice of Lisp to implement a language feature you use in another language. It doesn't matter if that language feature already exists in your choice of Lisp, you can still implement it yourself. For example, you can choose to implement C-style for loops or while loops, asynchronous coroutines like Go, pattern matching, lambdas, whatever. I actually implemented asnyc/await in IronScheme and pushed it upstream[0].
If you want to read more about Lisp macros, I have really enjoyed the book Let over Lambda. I have also heard a lot about On Lisp by pg, but I haven't read that myself yet. Also if you really want to dive off the deep end into the beauty of programming, I recommend SICP.
[0] https://github.com/IronScheme/IronScheme/pull/141
Been there, done that. The real challenge is not implementing a non-trivial macro; it's coming back to that non-trivial macro a week later.