The original Lemon parser generator, along with sibling projects, forks and assets — all in one place.
Project maintained by Tristano Ajmone since 2019/04/23.
Table of Contents
The Lemon program is an LALR(1) parser generator, written in C89 by Richard Hipp during the late 1980s. It takes a context free grammar and converts it into a subroutine that will parse a file using that grammar. Lemon is similar to the much more famous programs "YACC" and "BISON", but it's not compatible with them. There are several important differences that distinguish Lemon from Yacc and Bison:
- Lemon uses a different context free grammar syntax, which is less prone to programming errors.
- The parser generated by Lemon is both re-entrant and thread-safe.
- In Yacc and Bison, the parser calls the tokenizer (aka push parsing model). In Lemon, it's the tokenizer that calls the parser (aka pull parsing model).
- Lemon includes the concept of a non-terminal destructor, which simplifies writing parsers that don't leak memory.
- Lemon doesn't use global variables. Yacc and Bison use global variables to pass information between the tokenizer and the parser.
- Lemon allows multiple parsers to be running simultaneously; Yacc and Bison do not.
Lemon's innovative design inspired Bison to embody some of these features in the course of time.
For a detailed history of Lemon, see the History of Lemon section in lemon/lemon.md.
The complete source code to the Lemon parser generator is contained in two files: lemon.c and lempar.c.
Currently, this repository contains only the official version of Lemon (public domain) taken from the SQLite project:
/lemon/— original Lemon sources from SQLite:/examples/— user-contributed examples./sliced/— de-amalgamated Lemon (split sources).lemon.c— parser generator program.lempar.c— template for the parser generator.lemon.md— documentation.
The examples/ subfolder contains third-party examples based on this version of Lemon (the examples are not from the SQLite project).
There are many variations of the original Lemon code circulating over the Internet, and it's quite common to find tutorials and examples that rely on tweaked versions of Lemon, often in conjunction with other tools (like re2c).
To avoid confusion, in this project all examples are kept together with the Lemon version for which they were designed. Different versions of Lemon (forks, ports, etc.) will be kept in separate folders, and each version will have its own examples/ subfolder.
The idea is to keep the Lemon Grove tidy and well structured, so that its users can easily distinguish which version of Lemon is where, and how to find examples for that specific version.
The sliced/ subfolder contains a de-amalgamated version of Lemon, i.e. the single lemon.c source file is split into separate modules (23 files), like it used to be originally.
Anyone interested in studying or porting the Lemon source code will find it easier to work on these individual modules, instead of the official single-file source (>5600 lines).
Since de-amalgamation is done via a custom tool (Lemon Slicer) and derived from the code found in the lemon/ folder, the Lemon sources in the sliced/ folder will always mirror the code from the lemon/ folder.
Whenever the latter is updated, the former gets updated too.
External links to useful third party Lemon-assets.
The official Lemon code, actively maintained as part of the SQLite project.
- Lemon homepage
- Lemon documentation
- Upstream Lemon sources on SQLite repository (Fossil):
Adaptations of Lemon that add new functionality or provide native integration with other tools.
Created by @renjipanicker in 2015, public domain.
Lemonex is an extension to the Lemon parser, developed to include a built-in lexer.
Lemon has been adapted to produce parser generators in other programming languages too, by tweaking its C source code and creating an ad hoc version of lempar.c in the target language.
By Roopesh Chander in 2017, MIT licensed.
Citron is an LALR parser generator for Swift, based on the Lemon engine.
Created by @nsf in 2010, public domain.
Outdate and no longer maintained, but a good starting point for anyone willing to revive the project.
Created by @gwenn in 2017, released into the public domain via Unlicense.
A fork of the LEMON parser generator that generates Java code and the associated SQL parser.
Created by Artem Butusov in 2017, public domain.
Created by Wez Furlong in 2006, last updated in 2012, BSD-like license.
Created by Rodrigo Rivas Costa in 2015, released under Apache-2.0 license.
Lemon Rust is now deprecated in favour of pomelo, by same author.
Lemon implementations in other programming languages.
By Jeremiah Shaulov, released under MIT License.
Implementation of the Lemon parser generator as a Rust library with API.
By Rodrigo Rivas Costa, released under dual license — MIT or Apache-2.0.
Implementation of the Lemon parser generator as a Rust procedural macro.
Lemon has played an influential role in the development of parser generators, serving as a model for the creation of similar tools. Here are links to some of them.
By Martin Vejnár, Boost Software License 1.0.
limecc is a lexer and parser generator similar to other tools like Yacc, Bison and especially Lemon from which limecc sources inspiration. Grammars are written in a language called Lime, which describes lexical tokens, grammar productions, and semantic actions. The generator produces C++ code for the corresponding lexer and parser.
Packages/plugins for adding support for Lemon grammar files to various editors.
Missing packages? Please add them via pull request, or open an issue and provide a link.
By Yursen Kim, MIT License.
Adds syntax highlighting to Lemon Parser Generator files in Atom.
By Masafumi Oyamada, GNU GPL v3.
Emacs major mode for editing LEMON grammar files.
By @ksherlock, CC0 1.0 Universal.
Sublime Text 3 syntax file for the Lemon parser generator.
By Serghei Iakovlev, MIT License.
Lemon Parser Generator syntax highlighting for VSCode.
By @dccmx, MIT License.
Crappy syntax highlighting in Vim for Lemon Parser Generator grammars.
Links to some Lemon-related tools.
Lemon Slicer is a dedicated tool to de-amalgamate the "lemon.c" source file into separate modules.
It was created specifically for the Lemon Grove project.
By Tristano Ajmone, MIT License.
Links to useful books, articles and tutorials on the topics of lexing and parsing.
Free PDF book + sources, 984 pp.
Originally published in 1990 by Prentice-Hall Inc., Compiler Design in C, by Allen Holub, is an excellent book on the topic. Written in plain language, this book takes the reader through a 984 pages long journey on how to build a compiler, step by step, introducing and explaining each compiler component in detail, and providing source code examples of each implementation step.
Since the book is now out of print, the author has generously made it available for free download on his website, in PDF format, including all the source code files and the release notes.