Embracing modularity with Meson

Almost seven years ago, I made the first commit to my very first C library. It was a personal C library – so personal that I gave it the name “libstephen”, ensuring that nobody else would ever even consider using it. And to be fair, nobody should have used it! This was a data structures library written by a kid with a few weeks of C experience and a bunch of basic Java coursework under his belt. Without getting into specifics, the code was pretty bad.

But as bad as the code was, it was mine. It functioned correctly! It had tests! I was even able to start using it in other projects. Having pre-made data structures made C feel like an easier language. It let me tackle larger problems without drowning in the details of every little thing. But soon, a problem emerged. I invested a lot of time into making this library really easy to develop, but I didn’t know how to make it easy to use. The best way I knew to use libstephen was to create a git submodule, and then do something like this in your Makefile:

libstephen/bin/release/libstephen.a:
    make -C libstephen

CFLAGS += -I libstephen/include/
# Don't forget to add library to your link command

It was clumsy, but it worked. But I grew to hate this approach for a few reasons. First, there was no real versioning - if I made a breaking change in libstephen, I had to be sure not to update past a certain git revision until I updated my dependencies. And second, I had invested so much time in libstephen’s build system and tools, that making new projects was frustrating because they didn’t have any of those tools. Sure, I could add those features over and over to each new project, but that was repetitive busy work. I found a “better” way.

Rather than creating new libraries and new projects, I just fudged it: I threw all my code into this library. Want to implement regex? Boom! Put it in libstephen. A lisp-based programming language? Add it to libstephen! Logging library? Why not add it as well? What started out as a library of data structures got crazy bloated, all because (a) it wasn’t easy to create new projects, and (b) it wasn’t easy to use my existing libraries in projects.

As a result, libstephen has turned into a bit of an embarrassment for me. It’s very bloated, the code isn’t great, but it has a lot of useful stuff, and my projects depended on it. But a few months ago I stumbled upon a set of tools with the potential to solve this problem, and I’ve been using them to fix up this mess. The centerpiece to it all is build system called Meson.

Dependency management in Meson

For those who are into open source C/C++ projects, Meson is not new. Projects like GNOME and Systemd have migrated to Meson, and migrating software to Meson seems to still be a trendy thing to do.

For those who aren’t familiar, Meson is a build system similar to Autotools or CMake. It aims to solve a few problems:

1. Compiling your code (probably the easiest of the problems)
2. Finding libraries you depend on, and linking against them
3. Allowing users to configure your build

Vanilla Makefiles, as I had been most familiar with before this, are good at #1 but don’t help you much with #2 or #3. As a result, systems like CMake and Autotools will actually handle #2 and #3, and generate makefiles to do #1.

Coming from languages like Python, which have package management built-in, it almost seemed strange that C had no standard system for that. The closest thing to traditional package management in C is that you can compile many autotools projects like this:

tar xf package.tar.gz
cd package
./configure --add some --options here
make
sudo make install

Of course, this will fail if you’re missing a dependent library, so you need to do dependency resolution yourself. So at the end of the day, it’s not very easy, and it’s certainly not very automated.

Meson presents a pretty elegant solution to this whole mess. In your meson.build file (roughly similar to a Makefile), you write something like this:

libstephen_dep = dependency(
  'libstephen',
  fallback: ['libstephen', 'libstephen_dep'],
  version: '>=0.3.1',
)
# ...
my_project = executable(
  # ...
  dependencies : [
    libstephen_dep,
  ],
)

The dependency() function call basically says this:

1. Look for a C library named libstephen installed to the system.
2. If you find one, and it satisfies the version requirements, then use it!
3. Otherwise, fall back to a “subproject” called libstephen, and use the dependency named libstephen_dep declared in its meson.build.

Later, the executable() call declares a program, and by including the dependency, it handles linking the library with your program, as well as adding the correct include directories for you. Pretty nifty!

The “subproject” exists as a fallback for when the dependency isn’t installed to the system, but it’s actually pretty useful on its own. Essentially, your project contains a subprojects directory, which would contain a file like this named libstephen.wrap:

[wrap-git]
directory = libstephen.git
url = https://github.com/brenns10/libstephen
revision = v0.4.0

This instructs Meson to pull a git repository at a particular revision. You can also ask it to grab a tarball from the web (and provide a checksum for that file). The resulting directory should have a meson.build file which can build the project. If the project doesn’t use Meson, you can even add “patches” to the subproject which contain a meson.build. At the end of the day, your meson build file just needs to provide a “dependency” declaration that allows this library to get linked into your project.

When all this is in place, you can compile your project with:

meson build     # analogous to ./configure
ninja -C build  # analogous to make

Meson will automatically locate your dependencies and set up the build system to either link against the system version, or build a local version.

Transitive dependencies and diamonds

So what if project A depends on B, and B depends on C? Well, Meson makes it pretty simple. Project A needs to include subprojects for B and C. If project A included project B and didn’t know about C, Meson even provides a nice warning message:

|Looking for a fallback subproject for the dependency C
|subprojects/B/meson.build:26: WARNING: Dependency C not found but it is available in a sub-subproject.
|To use it in the current project, promote it by going in the project source
|root and issuing the following command:
|meson wrap promote subprojects/B/subprojects/C.wrap

This is my favorite kind of error message: one which tells you exactly how to fix it. Since project B already had the wrap file necessary to compile project C, Meson helpfully can copy it for you into project A’s subprojects directory. This is nice because a wrap file will specify the exact version of a dependency that you’re using, and you’re free to modify the version that B is using, or leave it alone.

This comes in handy for diamond dependencies. Take the following example:

A --- depends on -----> B ----- depends on -----\
 \                                               \
  \-- depends on -----> D ----- depends on -----> C

In this situation, we have A, B, and C in the same chain as last time. But in addition, A depends on D, and D also depends on C, creating a diamond. Although it seems like this is more complicated, the same principle applies! Project A will include the wrap file for B, C, and D, and B and D will share the same version of C. So long as there is a version of C which is compatible with both B and C, you’re all set. And, even better, if Meson finds any of the dependencies installed to the system, it will use those too.

Small projects and easy dependencies

This has really encouraged me to embrace modularity, and write small, simple libraries that can be easily tested and do one thing well. I’ve broken out many parts of the code from libstephen into smaller libraries which all share similar naming patterns and similar project structures:

• sc-regex
• sc-argparse
• sc-collections

Each one is much more focused (although the collections library could maybe be broken into smaller units) and can be included individually. It’s pretty easy to use each library, but in case I need a refresher on using the libraries, or how to install add them as a dependency, I’ve created the sc-examples repository which collects simple, fully-worked examples of small programs that depend on these libraries, with build scripts and everything.

To make it easier for me to create new modular libraries, I’ve created sc-template, which can simply be copied into a new repository to streamline things. None of the template files are too big or complicated, but this just streamlines the process even more. Finally, to collect all these projects into a unified whole, I’ve started using the Sourcehut project feature: sc-libs.

Embracing consistent tooling

Along with using Meson for all my C libraries, I’ve started standardizing on a few other tools which I use in my sc-template to enforce consistency:

• Meson automatically generates compile_commands.json, which can be consumed by clangd to provide a LSP server for any editor.
• I use a standard .clang-format file to enforce a roughly Linux-kernel coding standard on my code.
• I use the pre-commit library to automatically format my code when I commit.
• I’m starting to standardize on the Unity test framework, which integrates nicely into Meson’s testing system and automatic code coverage generation.
• All my projects use roughly semantic versioning scheme and maintain changelogs.

Looking forward, I hope that I can benefit from a few more pieces of shared tooling:

• Consistent use of Doxygen to document public APIs and produce documentation.
• Use a Sourcehut builds to do CI testing, and publish documentation as well as source tarballs for new releases.

The end result

The result of this work has been pretty great. After several months of (intermittent) effort, my IRC Chatbot cbot has been migrated onto Meson, and has managed to shake its libstephen dependency in favor of my sc-libs.

While there’s nothing too revolutionary about this set of tools I’ve been using, it has made me feel really refreshed. For the first time, I feel like I have a set of usable tools for managing C library dependencies, that enables me to write modular code like I would in any other language. Hopefully this long-winded post gives other some insight or inspiration to learn Meson too!

Legal • RSS



Stephen Brennan's Blog is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License