Lexer/Parser

Lexer could be said as Scanner that receives a text or file, one character at a time, as an input and attempts to match it with one of the lexical patterns (regular expressions). If matched, this pattern could be returned as a token to the parser. Parser reads the tokens and take action based on the rules.

In simpler words, lexing could be called as breaking the character stream into tokens and parsing could be called as analysing the relationship of the tokens.

Using lex and yacc tools, one could generate the corresponding c code or say, java code and use them in his program. YACC stands for Yet Another Compiler Compiler. By convention, a Yacc file has suffix .y. A lex file could have suffix such as .lex, or, .jflex (in case of java).

This technology has various useful applications. Lets say you want to make a web-crawler. That might require reading the content from the webpage and parsing the element href="...". Generating an appropriate lexer/parser could act as a boon to this process.

Various lexer and parser generators could be found at the following link:

Compiler tools