|
Leonardus
|
The project Leonardus coding and architectural documentation is split into
a part CODING covering notes on more general aspects of
programming that could also be applied in other C++ projects.
And these very specific notes.
Leonardus is an open source project.
There is an extensive catalog of automated tests. When making design
decisions, the testability of a possible solution is given high priority.
There is a constant refactoring of the code and documentation.
A backlog is maintained with GitLab issues tickets.
No technical debt should be accumulated. Corresponding issue tickets
always have high priority.
When encountering incidental findings (e.g., potential bugs, suboptimal
patterns, or missing edge-case handling) during development, create a GitLab
issue or task tag immediately. Even for minor observations. This ensures no
insight is lost and technical debt is preemptively addressed.
All artifacts can be built with a minimal toolset. Currently, these are
the GNU C++ compiler, make, flex, Bash and git together with the libraries
Boost, GMP and quadmath. The project is hosted on GitLab, and the
environment's features like CI/CD are used. However, a build of all
artifacts and tests must always be possible without GitLab.
Documentation is preferably embedded in the code and should be cleanly
extractable with Doxygen. All documentation is included in the git repository.
Readability and clarity of the program source code take precedence over
optimization in the implementation.
The project documentation and the source code are written in English.
For mathematical operators, whose creation is a primary goal of the project,
multilingualism is in the project backlog.
The main branch of the code must always allow a build of the artifacts
at the turn of the day, and these must pass the automated tests.
LeoScript will not have a garbage collection, but a controlled tree of all
existing semantic objects.
The execution of LeoScripts should be as reproducible as possible in terms
of runtime behavior. This means that:
LeoScript is designed to provide a conceptual and structural compatibility
with PostScript and Forth, though it does not aim for full syntactic or
functional equivalence.
LeoScript adopts the interpreter architecture of PostScript, but deliberately
excludes all graphics-related functionality. This approach already enables
non-graphical PostScript programs to run under LeoScript with minimal or no
modification. The focus is on the stack-based execution model and the language
semantics for algorithmic and computational tasks.
A further extension — currently in the project backlog — plans to introduce
dummy routines for PostScript’s graphical operations. This would allow
PostScript programs to be syntactically executable in LeoScript, even if
graphical output is not supported.
LeoScript does not aim for full Forth compatibility. However, it supports the
idiomatic style of Forth programming and allows small, typical Forth
algorithms to run unchanged in LeoScript. This is achieved by leveraging
Leonardus vocabulary concept — a system extension mechanism that allows
Forth’s "words" to be defined one-to-one in LeoScript.
IDEA: text
IDEA: text
This is a table of the SOs with details.
All objects are executable or non-executable aka literal.
This executable status becomes manifest with different OTCodes for
name objects and array objects.
For simple objects a duplicate of an object duplicates the value of the object.
For composite objects a duplicate of an object shares its value with the
original object.
There's an enum class OTCode in the source to list all the codes.
The column type shows the result of the operator type which is used to
identify objects in LeoScript code.
The column class shows the C++ class name of the SOs.
| objects | class | exe status | OTCode | type | description |
|---|---|---|---|---|---|
| simple | SOL | L | nulltype | The null object is used as placeholder in arrays. | |
| SOB | B | booleantype | A boolean value. | ||
| SOI | I | integertype | A 128-bit integer. | ||
| SOM | M | marktype | A mark object. | ||
| SON | literal | N | nametype | for non-executable name objects. | |
| executeable | Nx | nametype | for executable name objects. | ||
| SOO | O | operatortype | A regular registered code code operator. | ||
| SOo | o | operatortype | An unregistered core code operator. | ||
| SOR | R | realtype | A 128-bit IEEE real number. | ||
| SOQ | Q | rationaltype | A rational number with arbitrary precision. | ||
| composite | SOS | S | stringtype | A string. | |
| SOD | D | dicttype | A dictionary is a list of key-value pairs of SOs. | ||
| SOA | literal | A | arraytype | For a non-executable array of any SOs in any order. | |
| executable | Ax | arraytype | For an executable array aka procedure. | ||
| SOK | K | stacktype | A stack of SOs. |
The system also makes use of abbreviations (pseudo OTCodes) to determine
groups of OTCodes.
| abbreviation | description |
|---|---|
| Z | I or R or Q |
| X | any SO |
See TESTS
A development tool overview will be generated by the build process
as TOOLS.md.
Leonardus integrates the traditional UNIX man page subsystem.
Generated man pages are categorized under section 7 (Miscellaneous
Information Manual).
Doxygen generates man pages for the operators registered in:
systemdict,leodict anduserdict.The command lb -C manpage generates man pages for the operators from
vocabularies.
The tool mandb must be accessible in the PATH to update the database for apropos.
The man pages are
make install.There are three so-called build types. They denote build and, in particular,
compilation settings for different environments.
Compilation without time-consuming optimizations.
Used to develop and debug the code.
Compilation with good optimization. All debug-code is removed.
All asserts are removed.
Compilation without any optimization. No inlining of functions.
Compilation with instrumented code to generate profiling data.
Used to analyze function, line and branch test coverage.
make lcov only succeeds with this build type.
The GitLab pipeline uses the build types PRODUCTION and PROFILE.
The make utility can be directed by: BUILD_TYPE=PRODUCTION make leon or
as in the GitLab pipeline configuration by: make BUILD_TYPE=PRODUCTION leon
or by: export BUILD_TYPE=PRODUCTION.
If you change the build typing a make btclean is recommended.
The command line tools lc, lb, leon, and parser print the build type
with which they were generated as the first line in their help text.
These are: lb, lc, parser, leon.
These are: e2e.sh, ctest.sh.
The call and usage hierarchy (see hierarchy.drawio) results in the
segmentation of include statements into so-called IncBlocks.
The specific order of these blocks is as follows:
We use a two-part Version Number consisting of a major and minor
version, separated by a period. We started with "0.9".
Release Numbers are assigned to these version numbers by appending
a sequential number starting from 1. Therefore, the first release number
was 0.9.1. When the version number is increased, the last number is reset
to 1. Consequently, the first release of version 1.0 is 1.0.1.
Release numbers are stored in git as Release Tags prefixed with 'v'.
Thus, the first release tag is "v0.9.1". In git, the first commit intended
for a release is tagged with this release tag.
The last commit in the release cycle receives a Release Completion Tag
in git. This release completion tag in git has the form of the release tag
with the appended text "--release". Therefore, the first release
completion tag is named "v0.9.1--release".
For each release completion tag, a GitLab Release Object, i.e., a
release in the sense of GitLab, is created.
When a new release is created, a new section is opened in the
CHANGELOG.md for this release. The Release Date of the previous
release is also assigned then. This is the date the respective Git
release object was created.
The section of the previous release in the CHANGELOG is manually revised
when a new release is created. The revised content of the CHANGELOG section
for a release serves as the Release Notes.
For each commit, the corresponding commit message should be appended to
the CHANGELOG. This can be done automatically by installing a git hook
from TOOLS.
A release consists of:
customdocker.shgit pushgit push origin tag vx.y.z--releasegit pushgit push origin tag vx.y.z+1unset BUILD_TYPEmake cleanmake release-info outputmake e2e./leon -h release-info outputThe available makefile targets can be displayed by using the the
make command without options.
The makefile will be triggered from the GitLab pipelines. All process
details are implemented in the makefile and its helping scripts.
Intenionally there are no details coded in the pipeline YAML configuration.
Both the makefile and the CI/CD pipeline are structured into the same five stages:
The makefile supports two different dependency mechanisms:
The pipeline configuration utilizes parallel processing in GitLab runners by
invoking make -j$(nproc).
To achieve the same locally, you can run:
The project uses *.in template files (e.g., HTML, Bash, JSON) with
\@PLACEHOLDER\@ tokens, replaced at build time via sed. This ensures
version-controlled templates remain environment-agnostic, while generated
files (e.g., file.html, script.sh, config.json) embed runtime-specific values
(paths, versions, etc.). The approach is extensible. No hardcoded values are
committed.
Main interpreter loop
Interpreter::leonline().SON::load_exec() Processes one line of the leon-format input.
{, } are pushed onto Looks up a name and executes it.
Calls the C++ machine code associated with the SOO and SOo.
Unfolds duplicates of the array-content to the execution stack.
Replaces executable names with operator objects recursively into elements
that are SOA. Also does further optimization if Interpreter::odo_
ist set to true.
These dictionary stack manipulations influence what is found by
SON::load_exec().
The exec operator pushes
SOA::unfold2exec() andonto the execution stack.
The loop operators push
SOA::unfold2exec()onto the execution stack.
The installation of the Boost library in a GitLab pipeline image requires a
apt -y install gfortran- libboost-all-dev, because of Fortran config issues.
Following Boost modules are in use:
BOOST_ASSERT, BOOST_ASSERT_MSG, BOOST_ASSERT_IS_VOID
are used as building blocks for
DBC_PRE, BDC_POST, DBC_INV, DBC_INV_CTRO, DBC_INV_RAII and DBC_IS_VOID
See TESTS
GNU Multiple Precision Arithmetic Library (GMP) is a free library for
arbitrary-precision arithmetic, operating on signed integers, rational
numbers, and floating-point numbers.
In this project, we use the GMP library for the storage and algorithms of
rational numbers, specifically for the SOQ implementation. This is
complemented by an additional layer that provides adapters and
custom algorithms.
The quadmath library is a part of the GNU Compiler Collection (GCC) that
provides software implementations of mathematical functions for 128-bit
floating-point numbers (__float128). While the GCC supports the __float128
and __int128 data types natively, many standard library functions are not
available for these extended-precision types. The quadmath library fills this
gap by offering optimized implementations of essential mathematical
operations for __float128.
In Leonardus, the decision to use 128-bit floats and integers as fundamental
data types drives the need for quadmath. However, since not all required
functions are available out-of-the-box, the project includes an adapter128
module. This module acts as "glue code," bridging the gap between the native
__float128/__int128 support in GCC and the missing standard library
functionality.
The following GitLab features are used:
Additional features will be integrated over time.
KDevelop creates a .kdev4 file and a ./kdev4 directory. These files
are integrated into the git repository to share them.
VSC creates a ./vscode directory. This directory is integrated into
the git repository to share it as reference.
The following helpful adjustments can be made from the example configuration files c_cpp_properties.json und settings.json:
GITRELEASE and BUILD_TYPE.leo as PostScript fileThe 'Todo Tree' extension can handle our task tags appropriately.
The 'Lex' extension improves syntax highlighting for the flex source.
The 'Docker' extension supports Dockerfile editing.
There are three types of Docker images:
leonardusleojupyterleobuildAll images are stored on Docker hub (under the user hagenbund2).
The GitLab container regisitry is used exclusively to store the Leonardus
installation images.
The Dockerfiles to build the images are located in ./Docker.
The makefile target dockerize builds Docker images with a Leonardus
installation and pushes them to the Docker hub.
The target dockerforward copies the Leonardus images to the GitLab
container registry.
The Makefile generates two types of tags for installation images:
These tags are reassigned to new images when a new build is created:
| Tag | Description |
|---|---|
latest | Standard convention for the most recent build. |
release-<release number> | The last build of the given release. |
To ensure unique and immutable references, we use the Git commit hash in
the following formats:
| Tag Format | Description | Example |
|---|---|---|
CI-<build-type>-<commit-hash> | Generated in a GitLab pipeline. | CI-PRODUCTION-f3c904f |
DEV-<build-type>-<commit-hash> | Generated from a local developer command line. | DEV-PROFILE-f3c904f |
We build custom CI/CD images with preinstalled software primarily to speed up
GitLab pipelines.
The build steps are automated in the script customdocker.sh.
Below is a dependency tree of the Docker images, showing their origin and
FROM statements:
The Docker hub is used to store all images built with customdocker.sh and
from the makefile.
A manual clean up of old images has to be done frequently.
The Markdown-formatted "Overview" descriptions for Docker hub are stored as
templates in the directory ./AuxDoc.
The GitLab container registry is used exclusively to store images from the
makefile. It requires a slightly different naming scheme:
| Docker hub | GitLab container regisitry |
|---|---|
| hagenbund2/leonardus | hagenbund/leonardus/leonardus |
| hagenbund2/leojupyter | hagenbund/leonardus/leojupyter |
To run Docker in Docker on a workstation:
To run Docker in Docker on GitLab we use in the .gitlab-ci.yml:
$DOCKER_HUB_USER and $DOCKER_HUB_PASSWORD are stored as CI/CD variables in
the GitLab project.
$REGISTRY_USER and $CI_REGISTRY_PASSWORD are provided automatically by GitLab.
To run Docker commands as a non-root user, add your user to the "docker" group.
This is particularly necessary for integration with the makefile to trigger
the "dockerize" target locally during development. Use the following command:
Diagrams.net aka draw.io aka drawio is used for documentation.
This versatile tool allows users to create diagrams and export them to PDF.
For simplicity, the term "drawio" is used throughout the project to refer to
the program, file extension, directory, and related elements.
We use the DEB package v28.1.2 from GitHub jgraph/drawio-desktop/releases
to install drawio on development machines.
In the GitLab CI/CD the Docker image rlespinasse/drawio-export is applied.
Additionally, there is a valuable VS Code extension for drawio files
by publisher Henning Dieterichs.
Leonardus integrates with Jupyter, because LeoScript can be configured as a
Jupyter kernel.
The files kernel.json.in, kernel.py and logo-64x64.png in the directory
./Jupyter are the building blocks for a Jupyter kernel. They are installed
by make install to $HOME/.local or, in the case of the Docker image by
leojupyter.Dockerfile to /usr/local/share/jupyter/kernels.
kernel.py acts as an adapter, wrapping the lb interpreter to provide the
required kernel interface for JupyterLab.
The make install creates the necessary configuration to use Leonardus with
an existing JupyterLab installation.
An even tighter integration can be achieved using the Tools/udocker.sh
script. The script runs the Docker processes as the current login user,
enabling data sharing with Docker volumes.
You can open and work with *.ipynb notebooks from the project's ./Jupyter
directory using any Jupyter client, regardless of whether a kernel
configuration exists.
The leojupyter Docker image described by leojupyter.Dockerfile
contains all components needed to run JupyterLab directly from the image.
To start the JupyterLab X application, use the command provided in the
Docker hub repository's "Overview" section. It integrates with a
$HOME/leojupiter directory to provide persistent storage. It further
fine-tunes the startup of the X application by setting certain Docker security
options.
Ghostscript can be used as reference implemenation of the PostScript language.
Hint: To invoke gs without rendering, you can set the environment
variable export GS_DEVICE=nullpage.
1.9.8