<< Click to Display Table of Contents >> Navigation:  »No topics above this level« Components and files

The following is a brief description of some of the projects and files in the Gekko solution.

First a note on dynamic code:

 

There are the following projects in the Gekko solution (each project produces a .NET .dll file):

•alglib. This is a C# implementation of the ALGLIB linear algebra library. The source code is included in Gekko because it is sometimes beneficial to look into the source code of alglib.

•ANTLR. Gekko uses the ANTLR grammar files Cmd3.g (.gcm files) and Model.g (.frm files) to produce Gekko parsers generated in C#. These parsers stored in the ANTLR project.

•Deploy. This is for internal use, when deploying new Gekko versions (putting them on the Gekko website).

•Gekko. This is where all the main Gekko components reside.

•GekkoFlowChart. Gekko has some flowchart possibilities built in, regarding decomposition. This is not actively used at the moment (also uses the ZoomAndPan project).

•Numerics. This is Math.NET Numerics, with many useful mathematical methods. The source code is included in Gekko because it is sometimes beneficial to look into the source code.

•UnitTests. A large number of unit tests (regression tests) for testing out Gekko versions.

•ZoomAndPan. See the GekkoFlowChart project.

•InstallerForGekko. Installer project for producing an InstallerForGekko.msi file for easy installation.

 

For the Gekko project, there are the following files:

---

 
/Parser/ASTNode.cs. Used to store and handle an AST tree produced by the ANTLR parser. This particular AST node class is used for command files (.gcm).

/Parser/ASTNodeSimple.cs. Used to store and handle an AST tree produced by the ANTLR parser. This particular AST node class is used for model files (.frm).

/Parser/ParserCommon.cs. Nearly empty, but stores code used in both the /Frm and /Gek subfolders (model and command file parsing).

---

 
/Parser/Frm/ParserFrmCommon.cs. Empty at the moment.

/Parser/Frm/ParserFrmCompileAST.cs. Compile the dynamic C# code into an executable dll, and store the results in a protobuf internal file (cache) for later reuse. Compiles different versions of the code, useable for both Gauss-Seidel, Newton, and other uses. Also handles varlist (the variable list).

/Parser/Frm/ParserFrmCreateAST.cs. Mostly helper files to set ANTLR to parser the model file and produce an AST tree.

/Parser/Frm/ParserFrmWalkAST.cs. The most important part of the /Parser/frm folder: transforms the AST tree from ANTLR into runnable C# code. The code is a bit complicated, because the AST tree is actually converted into a DAG graph, in cases where the emitted C# is duplicated. For instance, Gekko code like the time-difference z = dif(x + y); is generated via first pointing to the node that produces C# code corresponding to x + y, and then re-pointing to the same node with a lag added. This operation on tree nodes complicates matters a bit, but is done like this to maximize parsing speed. Perhaps something like the more recent command file parser would be a simpler implementation (ParserGekWalkASTAndEmit.cs), and because of the caching of model C# code, parsing speed is not that important.

---

 

/Parser/Gek/ParserGekCommon.cs. Empty at the moment.

/Parser/Gek/ParserGekCompileAndRunAST.cs. Compile the dynamic C# code into an executable dll. Not that important.

/Parser/Gek/ParserGekCreateAST.cs. Mostly helper files to set ANTLR to parser the command file and produce an AST tree. Also, after the AST tree is walked, the C# code is split into smaller chunks (this part of the code ought perhaps to be moved to ParserGekCompileAndRunAST.cs).

/Parser/Gek/ParserGekWalkASTAndEmit.cs. The most important part of this folder: transforms the AST tree from ANTLR into runnable C# code. The ANTLR engine converts command files or command lines into a so-called AST tree. This tree is what Parser.cs gets as input, and the output from the methods in Parser.cs is C# code corresponding to the AST tree. The AST tree gets traversed in a recursive fashion, looking at a particular branch in the tree, and exploring its sub-branches etc. The AST tree itself is made by means of the methods in the project ANTLR. In particular, the AST tree corresponds to ANTLR syntax code in the file Cmd.g in the ANTLR project (these are syntax definitions).

---

 
Arrow.cs. These methods are for read/write of Apache Arrow files. This is experimantal and work in progress as of December 2020.

CrossThreadStuff.cs. The Gekko GUI is running threaded, so that it does not become unresponsive. When the worker thread needs to update the GUI (for instance change an icon etc.), we use Invoke() to be able to access another thread. All the methods use the same recipe, and the methods are easy to understand. It is practical to have all inter-thread communications stored in one file. (This module uses so-called C# delegates)

Databank.cs. This file defines a Gekko Databank class. As it is also the case regarding the Map class, the Databank class implements the IBank interface, to make sure that adding/getting/removing variables in databanks and maps work in the same way. This is because a map really is a kind of mini-databank. For instance, getting the variable (quarterly timeseries) x!q from the databank b has the syntax b:x!q, whereas getting x!q from the map #m has the syntax #m.x!q, but underneath this syntax difference, the inner workings of a databank and a map are very similar (using the same kinds of C# Dictionaries).

Databank_1_1.cs. The file defines a Databank class using the older 1.0 or 1.1 version of the databank format. This is legacy code, only used to be able to read/write protobuffer (.gbk) files or zipped tsd files (.tsdx) produced by Gekko versions < 3.0. The file also contains a TimeSeries_1_1 class, corresponding to how timeseries were implemented in Gekko versions < 3.0.

Databanks.cs. Definition of databank logic, that is, how the databanks are opened and closed in the databank hierarchy. The databank hierarchy is basically just a simple C# List, with the first-position databank as list member 0, and the special reference databank as list member 1.

DataFrame.cs. This is essentially empty, just making room for a later implementation of dataframes. The DataFrame class implements the IVariable interface. It is planned to implement dataframes via the new .NET Microsoft.Data.Analysis project.

Decomp.cs. The file contains all methods related to the Gekko DECOMP command, in its new version. Much more on the methods in this file here. The "old" DECOMP still resides in Program.cs, but the old DECOMP code will be obsolete and removed sooner or later.

EquationBrowser.cs stores everything related to producing a stand-alone equation browser for Gekko models. Gekko already has an inbuilt equation browser (activated with the DISP command), but this can only be used after downloading Gekko. The stand-alone browser produces a collection of html and graphics files that can be viewed and browsed with a normal Internet browser like Chrome or Edge. The files can be uploaded and pointed to with a simple internet link, easing access. The equation browser module uses some schema files and a .json settings file, but a user guide is unfortunately lacking at the moment.

Estimation.cs contains methods related to the OLS (ordinary least squares) command. In the longer run, other econometric methods may be added here.

Functions.cs. All in-built Gekko functions. These functions have IVariables as arguments and return an IVariable. Helper functions inside this file should either be private or start with Helper_, so that they are not identifies as "real" Gekko in-built functions. All these functions are in lower-case.

G.cs. The file contains a lot of (typically rather) small helper methods. The methods put into G.cs are typically of very general nature, that is, not expected to change a lot regarding inner workings or method signatures. They are also thought to be "general", useable from all parts of Gekko. Whether a method resides in G.cs or Program.cs can be a bit arbitrary.

Gams.cs. Inside this file, the GdxFast class handles read/write of gdx files, using the so-called "fast" low-level gdx interface.

GekkoNull.cs. A null variable, implementing the IVariable interface. Hence, Gekko functions can return null, Gekko lists can contain null values, etc.

GekkoTime.cs. Methods and objects/structs dealing with time periods and frequencies in Gekko. The GekkoTime class (which is in reality a struct) can iterate over periods, for instance quarters and months, and compare if one date is larger than or smaller than some other date. There are also converter methods to/from for instance C# DateTime objects, Excel datetime variables, etc. Also handles implicit conversions, for instance when print a quarterly timeseries, how to print it over the period 2020m5-2020m8 (May to August)? This period is implicitly converted into 2020q2-2020q3 (second and third quarter). More on time iteration here.

Genr.cs is just a helper file that can be autommatically created by Gekko if the Gekko parser emits C# with illegal syntax (for bug-tracking). Nothing important there.

Globals.cs contains a large number of global variables (settings) that are not intended to be changed by the user. Non-changing stuff that needs to be accessed from anywhere in Gekko is typically put there. There are (almost) no methods, only static variables/settings.

 
Gui.cs. This is the main Gekko user interface (GUI). The file defines menus etc., and also the two text components (upper part and lower part, corresponding to result and input window). Regarding the input window, there are quite a lot of methods for the handling of keystrokes, including the intellisense popup-window (cf. WindowIntellisense.xaml).

GuiCompareUtility3Way.cs, GuiCompareUtilityDatabanks.cs, GuiCompareUtilityEquations.cs. Three simple windows to handle different kinds of comparisons of databanks and models.

GuiDialogMakeBatfile.cs and GuiDialogMakeShortcut.cs are simple windows to produce a .bat file or shortcut (.lnk) file for easier Gekko startup. The former makes it possible to start up Gekko by simply writing gekko on the file prompt, and the latter puts a Gekko shortcut on the Windows desktop.

GuiGraph.cs. This contains everything related to the Gekko PLOT command, showing a graph (of timeseries).

GuiInputBox.cs. A simple inputbox corresponding to InputBox in VB.NET. Question is whether this component is used at all in Gekko?

GuiTspUtilities.cs and GuiTspUtilitiesData.cs. Utilities to handle data and estimation results from TSP.

IBank.cs. An interface that both Databank and Map classes implement. The interface ensures that databanks and maps have the same "feel" regarding getting/setting/removing variables. More about maps here.

IVariable.cs. An interface that makes it possible to treat "variables" equivalently in expressions. For instance, if x is a timeseries and %y is a value, Gekko will assess that the expression x + %y involves two IVariables (Series and a ScalarVal) together with an Add() method. To be more precise, in C#, the expression will look like var1.Add(var2), where var1 is the series, and var2 is the value. Therefore, C# calls the Add() method in the Series class, and the IVariable interface makes sure that whenever you use + to add two IVariables, the Add() method of the first variable gets called, with the second variable as argument. The IVariable interface is implemented for the Series, ScalarVal, ScalarDate, ScalarString, List, Map and Matrix variable types (object types).

Libraries.cs. Classes to handle the LIBRARY command, making it possible to put user-defined functions and procedures inside a zip-file (which works similarly to a R or Python package).

List.cs. A Gekko variable that implements the IVariable interface. A Gekko list contains an ordered sequence of other Gekko variables. Quite a lot of indexing, ranges, wildcards etc. is implemented in List.

ListViewDragDropManager.cs. This is used for the new DECOMP window, where fields can be dragged around in the pivot-like GUI interface.

Lookup.cs. Methods that deal with looking up (finding) a Gekko variable in a databank, and also storing results in a Gekko variable. More on lookups here.

LruCache.cs. A LRU cache is a Least-Recently-Used cache used for the caching of models in Gekko. There are two kinds of model caching, where the first is caching models to the file system. Here, all parsed model files are cached, but from time to time this storage is flushed. The other is keeping several model variations (of the same .frm file) in RAM: this is done when the same model for instance is changed via ENDO/EXO, remembering those variants. The capacity can be set with OPTION model cache max = ... ;, which is set to 20 as default. When the capacity is met, the least recently used model is booted from the cache, avoiding RAM overrun.

Map.cs. A Gekko variable that implements the IVariable interface. In addition it also implements the IBank interface, because a Map is also a kind of mini-databank. More on maps here.

Matrix.cs. A Gekko variable that implements the IVariable interface. Gekko implements a lot of matrix functionality, and Matrix can be used to store matrices. These are essentially 2d arrays of C# double values (64-bit numbers). So a Gekko matrix is numbers only, use nested lists for other variable types.

Model.cs is a container for storing all kinds of information related to a loaded model. At the moment only one model can be loaded in Gekko at a time, but the container will become practical when it will later on be possible to simulate different models at the same time. The Model class has a Model2 field (called .m2). This sub-object changes when for instance ENDO/EXO goals are set, and there is a RAM cache (Model2Cache) that stores these, cf. also LruCache.cs. More on models here.

O.cs. This contains quite a lot of methods, and some of them probably ought to be moved to Program.cs. The special thing about O.cs is that code snippets (in string form) that contain method calls generally only use static methods that can be found in O.cs. That way, when for instance refactoring or removing a method, if the method is a method from O.cs, special attention must be paid. If, for instance, you are refactoring O.MyMethod(), you should search for "O.MyMethod(" in the whole Gekko solution, because it may be used in dynamic code (composing C# code from strings). The file also contains a class corresponding to each Gekko command, for instance the class O.Clear, which handles the CLEAR command. Such classes are also called from dynamic code.

OnlineDatabanks.cs. This file deals with the DOWNLOAD command, downloading from a particular online databank (statbank.dk). More online databanks may be added later on.

Operators.cs. Methods that perform an operation on two different IVariable variables. For instance, adding the value 1.2 and a string 'a' produces 1.2a, whereas adding the string 'a' and the value 1.2 produces a1.2. But the internal logic is the same, so we provide a StringVal class method with optional invert parameter. This way, the logic can be stored in one place, instead of being implemented in both the ScalarString and ScalarVal classes. At the moment, only the Add() operator (+) is present in the file. More on operators here.

Options.cs. All the Gekko options (OPTION xx yy zz = ... ;) are found here. There is also code that deals with suggestions: the small popup-window ("Intellisense") that opens up, when the user types OPTION + [Enter]. There is also a method to print out all options (and their values) in the Gekko window. For instance, the value of the statement OPTION solve method = gauss; will be stored in the string variable Program.options.solve_method (which will contain the value "gauss"). The method Write() in Options.cs prints out current options. This is done by means of reflection, so Write() automatically prints out all options defined in Options.cs.

Plot.cs. All methods related to the Gekko PLOT command, including how the interface to gnuplot works.

Print.cs. Methods for printing variables on screen, especially timeseries and array-timeseries. More on printing here.

Program.cs. This is a large file is where a lot of the Gekko functionality resides, and it contains a large number of methods. A lot of the methods have to to with reading and writing different databank file formats, like .tsd, .csv., .xlsx, etc. Also, all kinds of methods that handle files and strings, etc. etc., and implementation of the different Gekko commands and in-built functions. Some of the more complicated commands like MODEL, SIM, PLOT, OLS, etc. have their own .cs files, but when a new method does not already belong to a particular .cs file, it is kind of "default" to put it into Program.cs. After refactoring, Program.cs is now much smaller than it used to be (now just large rather than enormous). Some smaller general methods reside in G.cs rather than Program.cs, and methods that are called from dynamic code reside in O.cs. But apart from that, Program.cs, G.cs and O.cs are similar, and mostly contain static methods and only quite simple classes/objects.

Range.cs. A special kind of class that implements the IVariable interface. You cannot define a Range object as an IVariable directy from Gekko code (something like #range = 1 .. 10;), but it is still being used in a variable-like manner inside Gekko. The only thing Range.cs can do is to create a new Range object from two IVariables (start, end). When for instance using #m[1..3] to fetch elements 1 to 3 of a list #m, it is practical that Gekko can "see" the range (that is, 1..3) as a special kind of variable, instead of seeing two variables (1 and 3) and dots (..). Otherwise there would be many combinations/permutations of dots and non-dots, for instance #m[1, 11, 21], #m[1..2, 11, 21], #m[1, 11..2, 21], #m[1, 11, 21..2], #m[1..2, 11..2, 21], and so on and so on.

RichTextBoxEx.cs is the "extended" rich text input textbox in the main Gekko GUI (the upper part of the window). Rich text is an old/obsolete format, and some day, when the main Gekko GUI is migrated to using .NET WPF instead of .NET WinForms, this upper window will probably become a WebBrowser control or something similar. It needs to be able to handle links, but the question if the WebBrowser control is fast enough for smooth scrolling etc.?

ScalarDate.cs. A Gekko variable that implements the IVariable interface. A scalar date contains a Gekko-date, defined in a specific frequency, like %d = 2020q2; (second quarter of 2020). A date can have an (integer) number of periods added or subtracted, which is handled in the class.

ScalarString.cs. A Gekko variable that implements the IVariable interface. This class stores a normal string inside, like %s = 'abc';.

ScalarVal.cs. A Gekko variable that implements the IVariable interface. This class stores a double-precision (64-bit) value inside. There are methods dealing with mathematical operations inside.

Series.cs. A Gekko variable that implements the IVariable interface. Series deals with creating and changing data in timeseries, that is, consecutive data in the time dimension. A timeseries has a frequency, and periods can be iterated over the GekkoTime periods, using a GekkoTimeIterator, and inside the timeseries, data is stored as an array of double-precision (64-bit) values. There is a special kind of series, namely light series, which is used to store intermediate results (like in for instance y = x1 * (x2 + x3);, the result x2 + x3 will be stored in a temporary light series that is fast to create). There are methods for the following: create normal series, create timeless series, create array-series, create light series, truncate series. Also methods for functions like log(x): ArithmeticsSeries(), methods for lag functions like pch(x): ArithmeticsSeriesLag(), methods for operating on a series with a value like x + %v: ArithmeticsSeriesVal(), methods for operating with two series like x + y: ArithmeticsSeriesSeries(), methods for operating with an array-series and a value like x + %v: ArithmeticsArraySeriesVal(), methods for operating with an array-series and a series like x + y: ArithmeticsArraySeriesSeries(), methods for operating with a series and an array-series like x + y: ArithmeticsSeriesArraySeries(), and methods for operating on two array-series: ArithmeticsArraySeriesArraySeries(). More on timeseries here.

Settings.cs. Whether or not this is used by UserSettings.cs or where it is used is an open question...

Solve.cs. This file contains modules related to model solving (the Gekko SIM command). The Gekko model parser is found inside the subfolder /Parser/Frm (for instance the file ParserFrmWalkAST.cs), so Solve.cs is not about parsing and compiling a Gekko model. It is more about solving the model, with the Gauss-Seidel algorithm, the Newton algorithm, or handling leaded variables (forward-looking models). More on solving here.

SplitContainerFix.cs. This class is a workaround for a but in the C# SplitContainer component. The splitter is used in the main Gekko GUI window, to split the upper and lower part of the window.

Stringlist.cs. Stores methods that handle lists of IVariables and lists of strings. Also contains methods that convert a List<string> into a a flat string with line breaks and vice versa. Converting a List<string> to a flat string with comma-separated parts is also supported.

StringTokenizer.cs is a quite advanced tokenizer. It builds upon a normal tokenizer that tries to split into words, symbols, strings, and whitespace. This is enhanced with the concept of "leftblanks", that is, for each token, how many blanks (possibly 0) there are to the left of the token. This is very practical when something is translated, and we need to try to keep blanks and indentation. The most advanced method is GetTokensWithLeftBlanksRecursiveHelper(), which creates a tree structure, where stuff inside (...), or [...], or {...} is put into a sub-tree. This makes it much easier to comprehend nesting structure while parsing. The recursive tokenizer is used for quite a lot of complicated stuff, like translating from older Gekko's or AREMOS to Gekko 3.0, or handling and comprehending GAMS model files. When a syntax or translation is not too complicated, it is simply much easier to user a recursive tokenizer, rather than using a real parser like ANTLR. More on tokenizers here.

Table.cs. The file contains all the Gekko table logic and formatting, cf. the Gekko TABLE command. Tables are usually defined using a xml table syntax, and can be "produced" as either text or html format (tables usually show timeseries). Borders, alignment, merge of cells, number formatting. Cells can generally store values, dates, or strings. To handle borders in text format, the whole table object is copied (cloned), which is perhaps a convoluted way of doing it. But the idea is to transform/blow up for instance a 2 x 3 table into a 5 x 7 table, where the "empty cells" around the real 2 x 3 cells can be used to put borders into. The PRT command and others use a Table object internally (text version), but note that there is also a TableLight class in Gekko, which is an ultra light-weight version of the full Table class (TableLight is used as intermediate interface for read/write of for instance csv, prn, xlsx files and others). More on tables here.

Task.cs. This small class is used for the databank window (F2), and for the DECOMP window. The Task class is used as rows in a kind of listview. It has to do with the fact that the MVVM pattern is used for these graphical windows.

Translate.cs. Methods to translate from Gekko 2.0 or AREMOS to Gekko 3.0. The translators use a recursive tokenizer in a quite advanced way. These translators are work in progress, but deal with some of the most tedious differences in syntax.

TspUtilities.cs contains utilities to convert data and equations from TSP to Gekko form. These utilities are accessible from the menu "Utilities" --> "TSP utilities".

UserSettings.cs stores different user settings, not intended to be explicitly stated by means of Gekko commands (in contrast to Options.cs). In contrast to user options in Options.cs, these settings get stored locally on the user pc and will be reloaded if the Gekko application is closed and restarted. At the moment, these settings deal with window sizes and positions, and the list of latest used working folders. These settings can be reset by means of the menu item "Options" --> "Restore user settings..." (or by deleting the xml file containing the data, see "Help" --> "About"). The settings should survive when upgrading to a newer Gekko version. Whether or not UserSettings.cs uses Settings.cs is an open question...

 

Window1.xaml is the "old" (existing) DECOMP window.

Window2.xaml is a simple error dialog window.

WindowDecomp.xaml is the "new" DECOMP window.

WindowDecompSortEtc.xaml is a helper window for sorting and showing variables.

WindowEquationBrowser.xaml. This is a new window, not in use yet (as of January 2021). It is related to WindowDecomp.xaml, and shows all equations a given variable is part of.

WindowIntellisense.xaml. A small popup-window used as visual helper when selecting options with OPTION xx yy = zz;.

WindowMessageBox.xaml. A generic MessageBox-style window for short messages and dialog.

WindowOpenDatabanks.xaml. The databank list (hierarchy overview), started with F2 from the Gekko GUI.

WindowRunStatus.xaml. The run status window, used to show the progress of Gekko jobs (used for larger systems of command files).

WindowTreeViewWithCheckBoxes.xaml. A window that supports a treeview of names, where the nodes in the tree have checkboxes. Used in the "new" DECOMP.

Wrap.cs. Classes that deal with printing text in the main Gekko GUI window. The classes are used for printing out normal messages, notes, warnings, and errors. Text can be wrapped around a given line width, and also be indented. There are a lot of convenience methods here, not least regarding error messages. Note that Wrap is an abstract "super-class", used by the sub-classes Writeln (normal text), Note, Warning and Error. These can be used either stand-alone, or via using. For instance new Note("Be careful!"); . But you may alternatively use using (Note note = new Note()) {note.MainAdd("Be careful!")}. The two snippets produce the same result, but when the printing is more involved, using is often used. Almost all warning and error messages has been ported to use the Warning or Error object, and in the long run, the older G.Writeln() methods will become obsolete.