5.2. How CMake Is Implemented
As mentioned, CMake's development languages are C and C++. To explain its internals this section will first describe the CMake process from a user's point of view, then examine its structures.
5.2.1. The CMake Process
CMake has two main phases. The first is the "configure" step, in which CMake processes all the input given to it and creates an internal representation of the build to be performed. Then next phase is the "generate" step. In this phase the actual build files are created.
Environment Variables (or Not)
In many build systems in 1999, and even today, shell level environment
variables are used during the build of a project. It is typical that a
project has a PROJECT_ROOT environment variable that points to the
location of the root of the source tree. Environment variables are
also used to point to optional or external packages. The trouble with
this approach is that for the build to work, all of these external
variables need to be set each time a build is performed. To solve this
problem CMake has a cache file that stores all of the variables
required for a build in one place. These are not shell or environment
variables, but CMake variables. The first time CMake is run for a
particular build tree, it creates a CMakeCache.txt file which
stores all the persistent variables for that build. Since the file is
part of the build tree, the variables will always be available to
CMake during each run.
The Configure Step
During the configure step, CMake first reads the CMakeCache.txt
if it exists from a prior run. It then reads CMakeLists.txt,
found in the root of the source tree given to CMake. During the
configure step, the CMakeLists.txt files are parsed by the
CMake language parser. Each of the CMake commands found in the file is
executed by a command pattern object. Additional CMakeLists.txt
files can be parsed
during this step by the include and add_subdirectory
CMake commands.
CMake has a C++ object for each of the commands that can be used in
the CMake language. Some examples of commands are add_library,
if, add_executable, add_subdirectory, and
include. In effect, the entire language of CMake is implemented
as calls to commands. The parser simply converts the CMake input files
into command calls and lists of strings that are arguments to
commands.
The configure step essentially "runs" the user-provided CMake
code. After all of the code is executed, and all cache variable values
have been computed, CMake has an in-memory representation of the
project to be built. This will include all of the libraries,
executables, custom commands, and all other information required to
create the final build files for the selected generator. At this
point, the CMakeCache.txt file is saved to disk for use in
future runs of CMake.
The in-memory representation of the project is a collection of
targets, which are simply things that may be built, such as
libraries and executables. CMake also supports custom
targets: users can define their inputs and outputs, and provide
custom executables or scripts to be run at build time. CMake
stores each target in a cmTarget object. These objects are
stored in turn in the cmMakefile object, which is basically a
storage place for all of the targets found in a given directory of the
source tree. The end result is a tree of cmMakefile objects
containing maps of cmTarget objects.
The Generate Step
Once the configure step has been completed, the generate step can take
place. The generate step is when CMake creates the build files for the
target build tool selected by the user. At this point the internal
representation of targets (libraries, executables, custom targets) is
converted to either an input to an IDE build tool like Visual Studio,
or a set of Makefiles to be executed by make. CMake's internal
representation after the configure step is as generic as possible so
that as much code and data structures as possible can be shared
between different built tools.
An overview of the process can be seen in Figure 5.1.
![[Overview of the CMake Process]](../../static/cmake/process.png)
Figure 5.1: Overview of the CMake Process
5.2.2. CMake: The Code
CMake Objects
CMake is an object-oriented system using inheritance, design patterns and encapsulation. The major C++ objects and their relationships can be seen in Figure 5.2.
![[CMake Objects]](../../static/cmake/objects.png)
Figure 5.2: CMake Objects
The results of parsing each CMakeLists.txt file are stored in
the cmMakefile object. In addition to storing the information
about a directory, the cmMakefile object controls the parsing
of the CMakeLists.txt file. The parsing function calls an
object that uses a lex/yacc-based parser for the CMake language.
Since the CMake language syntax changes very infrequently, and lex and yacc
are not always available on systems where CMake is being built, the
lex and yacc output files are processed and stored in the
Source directory under version control with all of the other
handwritten files.
Another important class in CMake is cmCommand. This is the base
class for the implementation of all commands in the CMake
language. Each subclass not only provides the implementation for the
command, but also its documentation. As an example, see the
documentation methods on the cmUnsetCommand class:
virtual const char* GetTerseDocumentation()
{
return "Unset a variable, cache variable, or environment variable.";
}
/**
* More documentation.
*/
virtual const char* GetFullDocumentation()
{
return
" unset(<variable> [CACHE])\n"
"Removes the specified variable causing it to become undefined. "
"If CACHE is present then the variable is removed from the cache "
"instead of the current scope.\n"
"<variable> can be an environment variable such as:\n"
" unset(ENV{LD_LIBRARY_PATH})\n"
"in which case the variable will be removed from the current "
"environment.";
}
Dependency Analysis
CMake has powerful built-in dependency analysis capabilities for individual Fortran, C and C++ source code files. Since Integrated Development Environments (IDEs) support and maintain file dependency information, CMake skips this step for those build systems. For IDE builds, CMake creates a native IDE input file, and lets the IDE handle the file level dependency information. The target level dependency information is translated to the IDE's format for specifying dependency information.
With Makefile-based builds, native make programs do not know how to
automatically compute and keep dependency information up-to-date. For
these builds, CMake automatically computes dependency information for
C, C++ and Fortran files. Both the generation and maintenance of these
dependencies are automatically done by CMake. Once a project is
initially configured by CMake, users only need to run make and
CMake does the rest of the work.
Although users do not need to know how CMake does this work, it may be
useful to look at the dependency information files for a project. This
information for each target is stored in four files called
depend.make, flags.make, build.make, and
DependInfo.cmake. depend.make stores the dependency
information for all the object files in the directory.
flags.make contains the compile flags used for the source files
of this target. If they change then the files will be recompiled.
DependInfo.cmake is used to keep the dependency information
up-to-date and contains information about what files are part of the
project and what languages they are in. Finally, the rules for
building the dependencies are stored in build.make. If a
dependency for a target is out of date then the depend information
for that target will be recomputed, keeping the dependency information
current. This is done because a change to a .h file could add a new
dependency.
CTest and CPack
Along the way, CMake grew from a build system into
a family of tools for building, testing, and packaging
software. In addition to command line cmake, and the CMake GUI
programs, CMake ships with a testing tool CTest, and a packaging tool
CPack. CTest and CPack shared the same code base as CMake, but are
separate tools not required for a basic build.
The ctest executable is used to run regression tests. A project
can easily create tests for CTest to run with the add_test
command. The tests can be run with CTest, which can also be used to
send testing results to the CDash application for viewing on the web.
CTest and CDash together are similar to the Hudson testing tool. They
do differ in one major area: CTest is designed to allow a much more
distributed testing environment. Clients can be setup to pull source
from version control system, run tests, and send the results to
CDash. With Hudson, client machines must give Hudson ssh access to the
machine so tests can be run.
The cpack executable is used to create installers for
projects. CPack works much like the build part of CMake: it interfaces
with other packaging tools. For example, on Windows the NSIS packaging
tool is used to create executable installers from a project. CPack
runs the install rules of a project to create the install tree, which
is then given to a an installer program like NSIS. CPack also
supports creating RPM, Debian .deb files, .tar,
.tar.gz and self-extracting tar files.
5.2.3. Graphical Interfaces
The first place many users first see CMake is one of CMake's user
interface programs. CMake has two main user interface programs:
a windowed Qt-based application, and a command line
curses graphics-based application. These GUIs are
graphical editors for the CMakeCache.txt file. They are
relatively simple interfaces with two buttons, configure and
generate, used to trigger the main phases of the CMake process. The
curses-based GUI is available on Unix TTY-type platforms and
Cygwin. The Qt GUI is available on all platforms. The GUIs can be
seen in Figure 5.3 and Figure 5.4.
![[Command Line Interface]](../../static/cmake/GUI1.png)
Figure 5.3: Command Line Interface
![[Graphics-based Interface]](../../static/cmake/GUI2.png)
Figure 5.4: Graphics-based Interface
Both GUIs have cache variable names on the left, and values on the
right. The values on the right can be changed by the user to values
that are appropriate for the build. There are two types of
variables, normal and advanced. By default the normal variables are
shown to the user. A project can determine which variables are
advanced inside the CMakeLists.txt files for the project. This
allows users to be presented with as few choices as necessary for a
build.
Since cache values can be modified as the commands are executed, the process of converging on a final build can be iterative. For example, turning on an option may reveal additional options. For this reason, the GUI disables the "generate" button until the user has had a chance to see all options at least once. Each time the configure button is pressed, new cache variables that have not yet been presented to the user are displayed in red. Once there are no new cache variables created during a configure run, the generate button is enabled.
5.2.4. Testing CMake
Any new CMake developer is first introduced to the testing process used in CMake development. The process makes use of the CMake family of tools (CMake, CTest, CPack, and CDash). As the code is developed and checked into the version control system, continuous integration testing machines automatically build and test the new CMake code using CTest. The results are sent to a CDash server which notifies developers via email if there are any build errors, compiler warnings, or test failures.
The process is a classic continuous integration testing system. As new code is checked into the CMake repository, it is automatically tested on the platforms supported by CMake. Given the large number of compilers and platforms that CMake supports, this type of testing system is essential to the development of a stable build system.
For example, if a new developer wants to add support for a new platform, the first question he or she is asked is whether they can provide a nightly dashboard client for that system. Without constant testing, it is inevitable that new systems will stop working after some period of time.