Why I'm Afraid of Systemd

Over the last two months or so it’s become apparent that systemd is taking over the Linux world. Starting with openSUSE and moving steadily through Fedora, Debian, Arch, RHEL, CoreOS, and even Ubuntu, systemd has taken over the Linux initialization process, slowly replacing sysvinit (or Upstart) as new versions of distributions become available, with one of the only hold-outs being Gentoo, which uses OpenRC. Many have expressed their reservations about systemd (including Linus Torvalds himself), but what will it mean for your business?

At DataXu we’re constantly on the front-line of changes in technology; new languages, whether Python, Ruby, Java, or Scala, enabling new libraries and including new features, drive our business to evolve our infrastructure regularly. Every update brings with it subtle challenges: new dependencies, deprecations, and many moving pieces make even minor upgrades a major orchestration. Developers have a tendency to ignore the realm of infrastructure, focusing instead on language features, and one thing often overlooked in a transition to a new major distribution of your chosen operating system is the way the system boots.

Right now the two operating systems that I use daily, CentOS at DataXu and Debian at home, are in the process of switching to systemd from sysvinit in their next stable versions (CentOS 7 and Debian 8 respectively). Sysvinit is the way most Linux operating systems, not to mention Unix and Unix-based operating systems like older versions of Mac OS, historically initialized the system at runtime and stopped it at shutdown since its inception in 1983 with the premier of SystemV. It’s compact, it’s well supported, and it’s time-tested. A rough estimate shows approximately 1200 Debian packages would need to be modified to work with other init systems, not to mention that custom init scripts required for in-house software and tools are likely to need updates.

So what’s wrong with sysvinit? It’s worked well for over 30 years! Well, it’s pretty old, meaning that it doesn’t do much besides manage the initialization process, and while many think that’s all init should do, it lacks the features that systems engineers all utilize independently in one form or another. It doesn’t restart daemons when they crash. It doesn’t understand hardware discovery. It’s not compatible with the “dynamic/event-based architecture of the current Linux kernel.”

Is systemd a reasonable alternative to sysvinit; are there favorable alternatives to systemd? Given its near ubiquitous adaptation it’s easy to assume that systemd is far and wide the superior choice for initialization for Linux, but why is the Linux community so divided, some rabid, over the decision to move major distributions like CentOS and Debian to systemd?

Let’s start by looking at the systemd architecture:

If it looks complex to you then you’re not alone. It has a crazy high code complexity with ~224K LOC (lines of code). For comparison Upstart has ~185K LOC, and sysvinit just ~5.8K LOC. The drastic difference in size is due to the addition of features like logind, journald, libraries for udev, even libraries for generating boot-chart graphs, and in general stuff that the Linux community prefers to have the choice to exclude. If you look at the Gentoo wiki on the comparison between init systems you’ll notice that systemd basically checks off the list of features, replacing a lot of the things that we’ve become used to dealing with in user-land. While this doesn’t eliminate choice of what to use, you cannot exclude these modules, only replace them as defaults post-install.

This creates what I would call a non-modular architecture. It utilizes binary logging (journald), replacing syslog, which, if used, means no more log grepping, log rotating, and so on. Cron disappears in favor of a proprietary cron called calendar timers with similar functionality. What was previously the responsibility of getty to log in now becomes the responsibility of logind. Systemd doesn’t work with other init managers and will not ever run on BSD (in the words of systemd founder BSD is no longer relevant). This raises the barrier to entry and the complexity of administering a Linux system, and it gives the impression of a Windows clone, locking a consumer into systemd. Pushing the barrier yet higher is the lack of compatibility for x86 or bare-dependency systems, essentially cutting out legacy support, something Microsoft would never do.

Systemd isn’t all bad, and there are some features I need to highlight to demonstrate the amount that’s lacking in sysvinit. It will provide configuration of device dependencies, built-in mount, fsck, automount, and swap handling. It has SELinux integration, PAM integration, and LUKS integration. It can handle locales seamlessly. It has the ability to respawn crashed services and detailed service analysis (such as listing all processes of a service). It has container support and stateful init scripts, so although you will lose complex init script chaining it should no longer be required. There truly are no alternatives that provide this level of support.

The only alternative worth mentioning (in my eyes) is OpenRC, which manages to supply many of the features above without some of the painful, inflexible tradeoffs. It is kernel agnostic, portable, and not so different from sysvinit as to be unrecognizable. In fact it does some things better than systemd, like having friendly upstream, friendly licensing, minimal dependencies, modular architecture, flexible scripts, and above all, the freedom of choice. Given that we are most of us already or will be shortly in the systemd boat I feel it is important to consider the alternatives carefully.