We had a few pretty cool courses in university touching this thing. It depends a bit on the work you want to do, but:
- Pick a language that's simple enough. A subset of ML would be good, but if you want to complete it, I'd recommend a simple LISP. This is your new language. This is C.
- Use a language you know and like to implement a compiler for this language. This is your bootstrap language. Compile this for example into C--, ASM or LLVM, depending on what you know. This is the target language. As a recommendation, keep this compiler as simple as possible so you have a reference for the next step. For C, both the bootstrap and the target language were ASM.
- And now iterate on extending the stdlib your language has, until you can implement a compiler for your new language in your new language. Again, keep this compiler simple without optimization or passes, just generate the most trivial machine code possible. This usually takes a bit of back-and-forth. You'll need some function evaluation first, some expression evaluation first (this is where a lisp can be an advantage, as those are the same), then you need function definitions, then you need filesystem interactions and so on. You kinda discover what you need as you implement.
- Once you have all of that, (i) compile the compiler for the new language in your bootstrap language and (ii) compile the compiler for the new language using the result of (i). If you want to verify the results, compile the compiler again with the output of (ii) and check if (ii) and (iii) are different.
- Your new language is now self-hosted.
This was fun, because it was accompanied with other courses like how processor microcode implements processor instructions, how different kinds of assembly is mapped onto processor instructions, and then how higher level languages are compiled down into assembly. All of this across 4-6 semesters resulted in a pretty good understanding how Java ends up being flip-flop operations.
EDIT - got target & bootstrap mixed up in first part.
A lot of people build toy languages too -- to the point that some become self hosting.
But there are also just an absolute ton of parsers out there to define a grammar and parse the language into an AST (Abstract Syntax Tree).
Once you have an AST then you can build an interpreter that runs the code directly in a JVM, or you can build a compiler that translates AST into LLVM, say.
Comments
We had a few pretty cool courses in university touching this thing. It depends a bit on the work you want to do, but:
- Pick a language that's simple enough. A subset of ML would be good, but if you want to complete it, I'd recommend a simple LISP. This is your new language. This is C.
- Use a language you know and like to implement a compiler for this language. This is your bootstrap language. Compile this for example into C--, ASM or LLVM, depending on what you know. This is the target language. As a recommendation, keep this compiler as simple as possible so you have a reference for the next step. For C, both the bootstrap and the target language were ASM.
- And now iterate on extending the stdlib your language has, until you can implement a compiler for your new language in your new language. Again, keep this compiler simple without optimization or passes, just generate the most trivial machine code possible. This usually takes a bit of back-and-forth. You'll need some function evaluation first, some expression evaluation first (this is where a lisp can be an advantage, as those are the same), then you need function definitions, then you need filesystem interactions and so on. You kinda discover what you need as you implement.
- Once you have all of that, (i) compile the compiler for the new language in your bootstrap language and (ii) compile the compiler for the new language using the result of (i). If you want to verify the results, compile the compiler again with the output of (ii) and check if (ii) and (iii) are different.
- Your new language is now self-hosted.
This was fun, because it was accompanied with other courses like how processor microcode implements processor instructions, how different kinds of assembly is mapped onto processor instructions, and then how higher level languages are compiled down into assembly. All of this across 4-6 semesters resulted in a pretty good understanding how Java ends up being flip-flop operations.
EDIT - got target & bootstrap mixed up in first part.
Do you have any books/online courses recommendations for learning what you just described?
A lot, if not all of this, was based on the Dragon Book[1]
https://en.wikipedia.org/wiki/Compilers:_Principles,_Techniq...
A lot of people build toy languages too -- to the point that some become self hosting.
But there are also just an absolute ton of parsers out there to define a grammar and parse the language into an AST (Abstract Syntax Tree).
Once you have an AST then you can build an interpreter that runs the code directly in a JVM, or you can build a compiler that translates AST into LLVM, say.
https://news.ycombinator.com/item?id=25388091