Almost seven years ago, I made the first commit to my very first C library. It was a personal C library – so personal that I gave it the name “libstephen”, ensuring that nobody else would ever even consider using it. And to be fair, nobody should have used it! This was a data structures library written by a kid with a few weeks of C experience and a bunch of basic Java coursework under his belt. Without getting into specifics, the code was pretty bad.
But as bad as the code was, it was mine. It functioned correctly! It had tests! I was even able to start using it in other projects. Having pre-made data structures made C feel like an easier language. It let me tackle larger problems without drowning in the details of every little thing. But soon, a problem emerged. I invested a lot of time into making this library really easy to develop, but I didn’t know how to make it easy to use. The best way I knew to use libstephen was to create a git submodule, and then do something like this in your Makefile:
libstephen/bin/release/libstephen.a: make -C libstephen CFLAGS += -I libstephen/include/ # Don't forget to add library to your link command
It was clumsy, but it worked. But I grew to hate this approach for a few reasons. First, there was no real versioning - if I made a breaking change in libstephen, I had to be sure not to update past a certain git revision until I updated my dependencies. And second, I had invested so much time in libstephen’s build system and tools, that making new projects was frustrating because they didn’t have any of those tools. Sure, I could add those features over and over to each new project, but that was repetitive busy work. I found a “better” way.
Rather than creating new libraries and new projects, I just fudged it: I threw all my code into this library. Want to implement regex? Boom! Put it in libstephen. A lisp-based programming language? Add it to libstephen! Logging library? Why not add it as well? What started out as a library of data structures got crazy bloated, all because (a) it wasn’t easy to create new projects, and (b) it wasn’t easy to use my existing libraries in projects.
As a result, libstephen has turned into a bit of an embarassment for me. It’s very bloated, the code isn’t great, but it has a lot of useful stuff, and my projects depended on it. But a few months ago I stumbled upon a set of tools with the potential to solve this problem, and I’ve been using them to fix up this mess. The centerpiece to it all is build system called Meson.
For those who are into open source C/C++ projects, Meson is not new. Projects like GNOME and Systemd have migrated to Meson, and migrating software to Meson seems to still be a trendy thing to do.
For those who aren’t familiar, Meson is a build system similar to Autotools or CMake. It aims to solve a few problems:
Vanilla Makefiles, as I had been most familiar with before this, are good at #1 but don’t help you much with #2 or #3. As a result, systems like CMake and Autotools will actually handle #2 and #3, and generate makefiles to do #1.
Coming from languages like Python, which have package management built-in, it almost seemed strange that C had no standard system for that. The closest thing to traditional package management in C is that you can compile many autotools projects like this:
tar xf package.tar.gz cd package ./configure --add some --options here make sudo make install
Of course, this will fail if you’re missing a dependent library, so you need to do dependency resolution yourself. So at the end of the day, it’s not very easy, and it’s certainly not very automated.
Meson presents a pretty elegant solution to this whole mess. In your
meson.build file (roughly similar to a Makefile), you write something like
libstephen_dep = dependency( 'libstephen', fallback: ['libstephen', 'libstephen_dep'], version: '>=0.3.1', ) # ... my_project = executable( # ... dependencies : [ libstephen_dep, ], )
dependency() function call basically says this:
libstepheninstalled to the system.
libstephen_depdeclared in its
executable() call declares a program, and by including the
dependency, it handles linking the library with your program, as well as adding
the correct include directories for you. Pretty nifty!
The “subproject” exists as a fallback for when the dependency isn’t installed to
the system, but it’s actually pretty useful on its own. Essentially, your
project contains a
subprojects directory, which would contain a file like
[wrap-git] directory = libstephen.git url = https://github.com/brenns10/libstephen revision = v0.4.0
This instructs Meson to pull a git repository at a particular revision. You can
also ask it to grab a tarball from the web (and provide a checksum for that
file). The resulting directory should have a
meson.build file which can build
the project. If the project doesn’t use Meson, you can even add “patches” to the
subproject which contain a
meson.build. At the end of the day, your meson
build file just needs to provide a “dependency” declaration that allows this
library to get linked into your project.
When all this is in place, you can compile your project with:
meson build # analogous to ./configure ninja -C build # analogous to make
Meson will automatically locate your dependencies and set up the build system to either link against the system version, or build a local version.
So what if project A depends on B, and B depends on C? Well, Meson makes it pretty simple. Project A needs to include subprojects for B and C. If project A included project B and didn’t know about C, Meson even provides a nice warning message:
|Looking for a fallback subproject for the dependency C |subprojects/B/meson.build:26: WARNING: Dependency C not found but it is available in a sub-subproject. |To use it in the current project, promote it by going in the project source |root and issuing the following command: |meson wrap promote subprojects/B/subprojects/C.wrap
This is my favorite kind of error message: one which tells you exactly how to
fix it. Since project B already had the wrap file necessary to compile project
C, Meson helpfully can copy it for you into project A’s
This is nice because a wrap file will specify the exact version of a
dependency that you’re using, and you’re free to modify the version that B is
using, or leave it alone.
This comes in handy for diamond dependencies. Take the following example:
A --- depends on -----> B ----- depends on -----\ \ \ \-- depends on -----> D ----- depends on -----> C
In this situation, we have A, B, and C in the same chain as last time. But in addition, A depends on D, and D also depends on C, creating a diamond. Although it seems like this is more complicated, the same principle applies! Project A will include the wrap file for B, C, and D, and B and D will share the same version of C. So long as there is a version of C which is compatible with both B and C, you’re all set. And, even better, if Meson finds any of the dependencies installed to the system, it will use those too.
This has really encouraged me to embrace modularity, and write small, simple libraries that can be easily tested and do one thing well. I’ve broken out many parts of the code from libstephen into smaller libraries which all share similar naming patterns and similar project structures:
Each one is much more focused (although the collections library could maybe be broken into smaller units) and can be included individually. It’s pretty easy to use each library, but in case I need a refresher on using the libraries, or how to install add them as a dependency, I’ve created the sc-examples repository which collects simple, fully-worked examples of small programs that depend on these libraries, with build scripts and everything.
To make it easier for me to create new modular libraries, I’ve created sc-template, which can simply be copied into a new repository to streamline things. None of the template files are too big or complicated, but this just streamlines the process even more. Finally, to collect all these projects into a unified whole, I’ve started using the Sourcehut project feature: sc-libs.
Along with using Meson for all my C libraries, I’ve started standardizing on a few other tools which I use in my sc-template to enforce consistency:
compile_commands.json, which can be consumed by clangd to provide a LSP server for any editor.
.clang-formatfile to enforce a roughly Linux-kernel coding standard on my code.
Looking forward, I hope that I can benefit from a few more pieces of shared tooling:
The result of this work has been pretty great. After several months of
(intermittent) effort, my IRC Chatbot cbot has been migrated onto Meson, and
has managed to shake its
libstephen dependency in favor of my
While there’s nothing too revolutionary about this set of tools I’ve been using, it has made me feel really refreshed. For the first time, I feel like I have a set of usable tools for managing C library dependencies, that enables me to write modular code like I would in any other language. Hopefully this long-winded post gives other some insight or inspiration to learn Meson too!