Implementing PEG in JavaScript

Parsing is an age-old technique used to analyze and extract meaning from languages (both natural and programming). Parser is a type of compiler that converts the stream of text into syntax or parse tree that conforms to some predefined grammar rules.

There are various classifications to categorize these techniques, and plenty of content is available to explain them. So, for now, I am focusing on Parser Expression Grammar (which is the most recent research in parsing grammar). Also, I will try to explain the ways to implement a PEG parser.

What Is PEG (Parser Expression Grammar)?

Parser Expression Grammar (PEG) is the formal grammar that defined a set of recursive rules under the family of top-down parsing language. Its parsers generate an explicitly single parse tree for any input. It is more powerful than regular expressions but might have some performance drawbacks related to memory and time in a few scenarios.

Advantages of Using PEG Parsers

PEG parsers have some advantages over other types of parsers. Most noticeably, they are unambiguous as they choose the first option among choices. Also, it is scanner less, which means it does not require a separate lexing phase. That makes it easier to implement for parsing needs that have smaller usability than that is required to parse a variety of inputs in an enterprise use case.

Understanding the PEG Structure

Let's try to understand the PEG structure with the following basic example that can be used for parsing arithmetic expressions.

start
  = additive

additive
  = left:multiplicative "+" right:additive 
  / multiplicative

multiplicative
  = left:primary "*" right:multiplicative 
  / primary

primary
  = integer
  / "(" additive:additive ")"

integer "integer"
  = digits:[0-9]+

Here, all the rules are recursive and drill down to literals or character classes with regular expressions. As we can see, an `additive` expression is an expansion of a `multiplicative` expression, and a `multiplicative` expression expands to an integer literal or nested additive expression. The integer literal is one or many occurrences of digits.

QuickStart

The quickest way to write or generate a PEG parser in javascript is to use pegjs. It is the most popular (as per GitHub) library to implement PEG parsers. You can refer to the official documentation for installation instructions. It supports both CLI and API modes for generating the parser.

Command Line

pegjs -o arithmetics-parser.js arithmetics.pegjs

JavaScript API

var peg = require("pegjs");

var parser = peg.generate("start = ('a' / 'b')+");

Online Tool

Apart from these modes, there is an online mode available that allows you not only to validate your grammar but also allows you to quickly test with sample inputs. Once you are done with testing, you can generate your parser on the fly with a speed or a code-optimized version.

Using the Parser

The generated parser can be used in both node and browser environments. You can call the `parse` method with test input, and it will either return a parse tree or an error ( for invalid inputs).

parser.parse("abba"); // returns ["a", "b", "b", "a"]

parser.parse("abcd"); // throws an exception

Sample Implementation

There is plenty of tools and services that use it in some or another way. The most update to date and advanced implementation is node-sql-parser (built by Zhi Tao). This is a pool of parsers for various modern query languages for databases like BigQuery, Hive, and Flink.