Details > Parser

The basic ideas of the parser has been explained in this and this section (and sub-sections).

The ANTLR grammars etc. are located in the /ANTLR sub-folder of the Gekko project.

Gekko commands

The ANTLR grammar used for command files is found in the file /ANTLR/Cmd3.g. This file describes the grammar, and the file is transformed into C# by means of the script MAKE_CMD3.bat. The script calls a Java .jar file (antlr-3.1.3.jar) together with Cmd3.g, which then tranforms it into the file Cmd3Lexer.cs and Cmd3Parser.cs. These files are used by Gekko for parsing of command files.

In /Parser/Gek/ParserGekCreateAST.cs, these files are used in the method ParseAndCallWalkAndEmit(). The ANTLR parser call looks something like this:

Cmd3Lexer lexer3 = new Cmd3Lexer(input);
CommonTokenStream tokens3 = new CommonTokenStream(lexer3);
Cmd3Parser parser3 = new Cmd3Parser(tokens3);
Cmd3Parser.start_return r3 = parser3.start();
CommonTree t = (CommonTree)r3.Tree;
ASTNode root = new ASTNode(null);
CreateASTNodesForCmd(t, root, 0);

The input string is called input, and first, a lexer (lexer3) is defined using this input. This lexer is fed to a token stream in the next line. In the third line, the tokens are fed to the parser (parser3). The parser is started in the fourth line, using parser3.start(). After this, the r3 object contains an AST tree, which is seen as CommonTree t above. This tree could be used for Gekko parsing, but we prefer to make a copy of it via the method CreateASTNodesForCmd(), creating a tree structure of ASTNode's. These ASTNode's contain Gekko-specific information, and are therefore easier to work with.

The ASTNode tree structure is used by /Parser/Gek/ParserGekWalkASTAndEmit.cs, which "walks" the nodes while emitting C# code.

Gekko models

Model parsing is in general simpler, because models allow less syntax. Models are basically more about implementing mathematical operators and the like, and therefore Model.g (the model ANTLR grammar) is a lot more simple than Cmd3.g for command files). Model.g can be "compiled" with MAKE_MODEL.bat, producing ModelLexer.cs and ModelParser.cs.

For model parsing, the ANTLR parser call looks a lot like before:

ModelLexer lexer = new ModelLexer(input);
CommonTokenStream tokens = new CommonTokenStream(lexer);
ModelParser parser = new ModelParser(tokens);
ModelParser.expr_return r = parser.expr();
t = (CommonTree)r.Tree;
ASTNodeSimple equationNode = new ASTNodeSimple(null);
CreateASTNodesForModel(t, equationNode, 0, wh, Program.model.modelGekko);

So when a model is parsed, a tree structure of ASTNodeSimple's are produced. The tree structure of ASTNodeSimple's is used as input in /Parser/Frm/ParserFrmWalkAST.cs, which transforms the nodes into C# code suitable for running models.

In general, /Parser/Frm/ParserFrmWalkAST.cs is much less complicated than /Parser/Gek/ParserGekWalkASTAndEmit.cs. The latter has to deal with conditionals, loops, user-defined functions/procedures, and a lot more.