The Dune Mental Model¶

It is not strictly necessary to understand Dune’s underlying model to use it; but knowing how it works under the hood will help writing build rules, and also help understand some errors and what’s possible with Dune.

Note

This document is a simplification of the reality: the actual rules might be different, it does not touch rule loading and glosses over how caching works, but should be a useful tool to build an understanding of Dune.

How Dune Works¶

The building block of Dune is the rule:

A rule reads dependencies and writes targets using an action (and it can be attached to aliases).

When dune build is executed, it will first read the project’s dune files to determine the rules that apply to the project. Once it has done this, it will determine what actions it needs to execute to build the required targets.

An Example¶

Let’s take the following example.

there’s a CLI tool written in OCaml.
it has some build-time configuration stored in config.json.
it has an integration test, in which the tool is executed with testdata.txt as input.

Configuration Generation¶

To express the generation of the configuration module we could write:

(rule
 (deps convert/json2ml.exe config.json)
 (target config.ml)
 (action
  (run convert/json2ml.exe config.json -o config.ml)))

This rule will:

read its dependencies: convert/json2ml.exe and config.json
and write its target: config.ml
using an action: (run convert/json2ml.exe config.json -o config.ml)

This rule is very explicit: we write a stanza for a single Dune rule.

Building the Executable¶

In contrast, to describe the compilation of the executable, we would write:

(executable
 (name tool)
 (modules main config))

Here, we use Dune’s abstractions. Dune knows about the OCaml compilation model: the modules need to be compiled and linked together. So it will generate the following rules under the hood:

one rule to compile the Main module:
- it will read its dependency: main.ml
- and write its output: main.cmx
- using an action: (run ocamlopt -c main.ml)
one rule to compile the Config module:
- it will read its dependency: config.ml
- and write its output: config.cmx
- using an action: (run ocamlopt -c config.ml)
one rule to link the tool.exe executable:
- it will read its dependencies: main.cmx and config.cmx
- and write its output: tool.exe
- using an action: (run ocamlopt -o tool.exe main.cmx config.cmx)

Note that in this example, some files are targets of a rule and dependencies of another (.cmx files). We are unlikely to ever interact with them directly, so it can also be useful to think of the (executable) stanza as a group of rules with main.ml and config.ml as inputs and tool.exe as output.

Running the Tests¶

Some rules do not produce any output file, but we’re still interested in running their actions. A test is a good example: we want the build process to exit with an error code if the action fails. In that case, the rule does not have targets, but we “attach” it to an alias, runtest in this case. This gives us a way of requesting this rule to be executed. As we are about to see, rules are executed lazily by asking for their targets to be built, so we would not be able to execute such rules.

(rule
 (deps tool.exe testdata.txt)
 (alias runtest)
 (action
  (run tool.exe testdata.txt)))

This rule:

reads its dependencies: tool.exe and testdata.txt
writes no targets
using an action: (run tool.exe testdata.txt)
(and it is attached to runtest)

What to Build¶

Dune can build files and aliases. These can be found on the command line:

dune build tool.exe will build the tool.exe file.
dune build @example will build the example alias.
dune build tool.exe @example will build both the file tool.exe and the example alias.
dune runtest is a shortcut for dune build @runtest: it will build the runtest alias. Passing a directory will build all tests in that directory. Passing the path to a cram test will run that test individually.
dune build is a shortcut for dune build @@default: it will build the default alias in the current directory (by default the all alias).

In other words, each dune build or dune runtest command always corresponds to a list of files and aliases to build.

How Dune Interprets Rules¶

We have now seen that Dune sets up rules for a project, and that every build command has a list of files and aliases that we are asking to build.

Now let’s see how this request is processed:

to build a file, Dune will first check if it is in the source tree. In that case, there is nothing to do. Otherwise, it will check if it is the target of a rule. In that case, it will execute this rule. (Dune will raise an error in other cases: if the file is both in the source tree and the target of a rule, or if it is neither)
to build an alias, Dune will execute all the rules that are attached to this alias.
to execute a rule, Dune will first build all the dependencies (files or aliases) of this rule. Then it will execute the action attached to the rule. When Dune is about to execute an action, it checks (in various caches) if it executed it before on the same set of dependencies, and, if yes, it can skip executing it and reuse the previous result.

In the case of our example, if we call dune runtest, Dune will consider all rules attached to the runtest alias. In this case it is just the integration test rule. It needs to build its dependencies, tool.exe and testdata.txt. The latter is present in the source tree. However, tool.exe is the target of the linking rule defined by the (executable) stanza. This rule requires main.cmx and config.cmx. main.cmx is the target of the compilation rule for the Main module, which depends on main.ml. This file is in the source tree, so let’s copy it under _build. This rule has all its dependencies available, so we can run its action, which writes main.cmx. Getting back to the dependencies of tool.exe, config.cmx is the target of the linking rule of the Config module. This rule has config.ml has a dependency. This file is itself the target of the configuration module rule, which lists config.json and convert/json2ml.exe. The first is available in the source tree and to simplify, let’s assume that the second one has been built. This action has all its dependencies available, so we can execute its action to produce its target, config.ml. Now the module compilation rule for Config can be executed, producing config.cmx; and in turn the linking rule can be executed, producing tool.exe. Finally, tool.exe can be executed with testdata.txt as its argument.

In a nutshell: we recursively copied all the dependencies of the test rule, and executed the rules in the correct order.

This is a “cold build”, where there were no previous build artifacts. Note that if we change only part of the project (say the main.ml file), only a small number of rules will be evaluated, the ones that depend on main.ml.

Conclusion¶

Dune’s underlying model is based on rules. Stanzas are high-level constructs that can generate multiple rules, that are not always visible.

To build a target, Dune looks for the rule that produces that target and makes its way back to source files.

Rules define a directed acyclic graph which models dependency relations between files. Most of the rules in that graph may be executed for a cold build, but just the minimum will be executed for an incremental build.