68 Resources on Creating Programming Languages, Part 2

In Part 1 of this series, we went over some resources to help you get started with designing a language and parsing. Today, we'll be looking at the best resources that deal with execution.

Execution

To implement your programming language, that is to say, to actually make something happen, you can build one of two things: a compiler or an interpreter. You could also build both of them if you wanted. Here you can find a good overview if you need it: Compiled and Interpreted Languages.

The resources here are dedicated to explaining how compilers and/or interpreters are built, but for practical reasons, they also often explain the basics of creating lexers and parsers.

Compilers

A compiler transforms the original code into something else, usually machine code, but it could also simply be any lower level language, such as C. In the latter case, some people prefer to use the term transpiler.

Tools

• LLVM. A collection of modular and reusable compiler and toolchain technologies used to create compilers.
• CLR is the virtual machine part of the .NET technologies that permits you to execute different languages transformed into a common intermediate language.
• JVM. The Java Virtual Machine that powers the Java execution.

Articles and Tutorials

• Building Domain Specific Languages on the CLR. This is an article on how to build internal DSL on the CLR. It’s slightly outdated since it’s from 2008, but it’s still a good presentation on the subject.
• The digital issue of MSDN Magazine for February 2008 (CHM format), contains an article on how to Create a Language Compiler for the .NET Framework. It’s still a competent overview of the whole process.
• Create a working compiler with the LLVM framework, Part 1 and Part 2, is a two-part series of articles on creating a custom compiler by IBM, from 2012 and is thus slightly outdated.
•  A few series of tutorials from the LLVM Documentation. This is a great series of three tutorials on how to implement a language, called Kaleidoscope, with LLVM. The only problem is that some parts are not always up-to-date.
• My First LLVM Compiler, a short and gentle introduction to the topic of building a compiler with LLVM.
• Creating an LLVM Backend for the Cpu0 Architecture. A whopping 600-page tutorial to learn how to create an LLVM backend, also available in PDF or ePub. The content is great, but the English is lacking. On the positive side, if you are a student, they feel your pain of transforming theoretical knowledge into practical applications, and the book was made for you.
• A Nanopass Framework for Compiler Education. A paper that presents a framework to teach the creation of a compiler in a simpler way, transforming the traditional monolithic approach into a long series of simple transformations. It’s an interesting read if you already have some theoretical background in computer science.
• An Incremental Approach to Compiler Construction (PDF). A paper that is also a tutorial that develops a basic Scheme compiler with an easier to learn approach.

Books

• Compilers: Principles, Techniques, and Tools, 2nd Edition. This is the widely known Dragon book (because of the cover) in the 2nd edition (purple dragon). There is a paperback edition, which probably costs less but it has no dragon on it, so you cannot buy that. It is a theoretical book, so don’t expect the techniques to actually include a lot of reusable code.
• Engineering a Compiler, 2nd edition. This is another compiler book with a theoretical approach, but it takes a more modern approach and it is more readable. It’s also more dedicated to the optimization of the compiler. So if you need a theoretical foundation and an engineering approach this is the best book to get.

Interpreters

An interpreter directly executes the language without transforming it into another form.

Articles and Tutorials

• A simple interpreter from scratch in Python. A four-parts series of articles on how to create an interpreter in Python; simple yet good.
• Let’s Build A Simple Interpreter. A twelve-parts series that explains how to create an interpreter for a subset of Pascal. The source code is in Python, but it has the necessary amount of theory to apply to another language. It also has a lot of funny images.
• How to write an interpreter. This is a screencast, with source code available, on how to write an interpreter for a simple language with Python.
• How to write a simple interpreter in JavaScript. A well-organized article that explains how to create a simple interpreter in JavaScript.

Books

• Writing An Interpreter In Go. Despite the title, it actually shows everything from parsing to creating an interpreter. It’s a contemporary book both in the sense that it's recent (a few months old), and it is a short one with a learn-by-doing attitude full of code, testing and without 3-rd party libraries. We have interviewed the author, Thorsten Ball.
• Crafting Interpreters. A work-in-progress and free book that already has good reviews. It is focused on making interpreters that work well, and in fact, you will build two of them during the course of the book. Its plan is to have just the right amount of theory to be able to fit in at a party of programming language creators.