What are the two system initialization processes utilized by linux to load and manage services?

Press the power button on your system, and after few moments you see the Linux login prompt.

Show

Have you ever wondered what happens behind the scenes from the time you press the power button until the Linux login prompt appears?

The following are the 6 high level stages of a typical Linux boot process.

What are the two system initialization processes utilized by linux to load and manage services?

1. BIOS

  • BIOS stands for Basic Input/Output System
  • Performs some system integrity checks
  • Searches, loads, and executes the boot loader program.
  • It looks for boot loader in floppy, cd-rom, or hard drive. You can press a key (typically F12 of F2, but it depends on your system) during the BIOS startup to change the boot sequence.
  • Once the boot loader program is detected and loaded into the memory, BIOS gives the control to it.
  • So, in simple terms BIOS loads and executes the MBR boot loader.

2. MBR

  • MBR stands for Master Boot Record.
  • It is located in the 1st sector of the bootable disk. Typically /dev/hda, or /dev/sda
  • MBR is less than 512 bytes in size. This has three components 1) primary boot loader info in 1st 446 bytes 2) partition table info in next 64 bytes 3) mbr validation check in last 2 bytes.
  • It contains information about GRUB (or LILO in old systems).
  • So, in simple terms MBR loads and executes the GRUB boot loader.

3. GRUB

  • GRUB stands for Grand Unified Bootloader.
  • If you have multiple kernel images installed on your system, you can choose which one to be executed.
  • GRUB displays a splash screen, waits for few seconds, if you don’t enter anything, it loads the default kernel image as specified in the grub configuration file.
  • GRUB has the knowledge of the filesystem (the older Linux loader LILO didn’t understand filesystem).
  • Grub configuration file is /boot/grub/grub.conf (/etc/grub.conf is a link to this). The following is sample grub.conf of CentOS.
  • #boot=/dev/sda default=0 timeout=5 splashimage=(hd0,0)/boot/grub/splash.xpm.gz hiddenmenu title CentOS (2.6.18-194.el5PAE) root (hd0,0) kernel /boot/vmlinuz-2.6.18-194.el5PAE ro root=LABEL=/ initrd /boot/initrd-2.6.18-194.el5PAE.img
  • As you notice from the above info, it contains kernel and initrd image.
  • So, in simple terms GRUB just loads and executes Kernel and initrd images.

4. Kernel

  • Mounts the root file system as specified in the “root=” in grub.conf
  • Kernel executes the /sbin/init program
  • Since init was the 1st program to be executed by Linux Kernel, it has the process id (PID) of 1. Do a ‘ps -ef | grep init’ and check the pid.
  • initrd stands for Initial RAM Disk.
  • initrd is used by kernel as temporary root file system until kernel is booted and the real root file system is mounted. It also contains necessary drivers compiled inside, which helps it to access the hard drive partitions, and other hardware.

5. Init

  • Looks at the /etc/inittab file to decide the Linux run level.
  • Following are the available run levels
    • 0 – halt
    • 1 – Single user mode
    • 2 – Multiuser, without NFS
    • 3 – Full multiuser mode
    • 4 – unused
    • 5 – X11
    • 6 – reboot
  • Init identifies the default initlevel from /etc/inittab and uses that to load all appropriate program.
  • Execute ‘grep initdefault /etc/inittab’ on your system to identify the default run level
  • If you want to get into trouble, you can set the default run level to 0 or 6. Since you know what 0 and 6 means, probably you might not do that.
  • Typically you would set the default run level to either 3 or 5.

6. Runlevel programs

  • When the Linux system is booting up, you might see various services getting started. For example, it might say “starting sendmail …. OK”. Those are the runlevel programs, executed from the run level directory as defined by your run level.
  • Depending on your default init level setting, the system will execute the programs from one of the following directories.
    • Run level 0 – /etc/rc.d/rc0.d/
    • Run level 1 – /etc/rc.d/rc1.d/
    • Run level 2 – /etc/rc.d/rc2.d/
    • Run level 3 – /etc/rc.d/rc3.d/
    • Run level 4 – /etc/rc.d/rc4.d/
    • Run level 5 – /etc/rc.d/rc5.d/
    • Run level 6 – /etc/rc.d/rc6.d/
  • Please note that there are also symbolic links available for these directory under /etc directly. So, /etc/rc0.d is linked to /etc/rc.d/rc0.d.
  • Under the /etc/rc.d/rc*.d/ directories, you would see programs that start with S and K.
  • Programs starts with S are used during startup. S for startup.
  • Programs starts with K are used during shutdown. K for kill.
  • There are numbers right next to S and K in the program names. Those are the sequence number in which the programs should be started or killed.
  • For example, S12syslog is to start the syslog deamon, which has the sequence number of 12. S80sendmail is to start the sendmail daemon, which has the sequence number of 80. So, syslog program will be started before sendmail.

There you have it. That is what happens during the Linux boot process.


What are the two system initialization processes utilized by linux to load and manage services?
What are the two system initialization processes utilized by linux to load and manage services?
What are the two system initialization processes utilized by linux to load and manage services?
What are the two system initialization processes utilized by linux to load and manage services?

Every time you start your Linux computer, its initialization system launches a batch of processes, from a few dozen to hundreds, depending on how the system is set up. You can see this on your startup screen (Figure 4-1; press the Escape key to hide your graphical startup screen and see the startup messages).

What are the two system initialization processes utilized by linux to load and manage services?

In olden times we had the Unix System V initialization system (SysV init), BSD init, and Linux Standard Base (LSB) init for launching processes at startup. SysV init was the most common. Those days are fading away, and now systemd is the shiny new init system for Linux. It has been adopted by all the major Linux distributions, though of course there are a number of distributions that still use the legacy init systems.

In this chapter you will learn if your Linux distribution uses systemd. You will learn what processes, threads, services, and daemons are, and how to use systemd to manage services: start, stop, enable, disable, and check status. You will become acquainted with the systemctl command, which is the systemd system and service manager.

systemd is designed to provide functionality suited to modern complex server and desktop systems, and it does considerably more than the legacy init systems. It provides complete service management from startup to shutdown, starting processes at boot, on-demand after boot, and shutting down services when they are not needed. It manages functions such as system logging, automounting filesystems, automatic service dependency resolution, name services, device management, network connection management, login management, and a host of other tasks.

This sounds like a lot until you realize that processes do everything on a computer, and all of this functionality used to be provided by a large assortment of other programs. systemd brings it all together in an integrated software suite that should operate the same way on all Linux systems, though as always with Linux there are some minor exceptions, such as file locations and service names. Be aware that your particular Linux may have some differences from the examples in this chapter.

systemd attempts to decrease boot times and parcel out system resources more efficiently by starting processes concurrently and in parallel, and starting only necessary services, leaving other services to start after boot as needed. A service that is dependent on other services no longer has to wait to start for those services to become available because all it needs is a waiting Unix socket to become available. Recipe 4.9 shows how to find processes that are slowing down your system startup.

systemd binaries are written in C, which provides some performance enhancement. The legacy inits are masses of shell scripts, and any compiled language operates faster than shell scripts.

systemd is backwards compatible with SysV init. Most Linux distributions retain the legacy SysV configuration files and scripts, including /etc/inittab and the /etc/rc.d/ and /etc/init.d/ directories. When a service does not have a systemd configuration file, systemd looks for a SysV configuration file. systemd is also backward compatible with Linux Standard Base (LSB) init.

systemd service files are smaller and easier to understand than SysV init files. Compare a SysV init file for sshd with its systemd service file. This is a snippet of the sshd init file, /etc/init.d/ssh, from MX Linux:

#! /bin/sh ### BEGIN INIT INFO # Provides: sshd # Required-Start: $remote_fs $syslog # Required-Stop: $remote_fs $syslog # Default-Start: 2 3 4 5 # Default-Stop: # Short-Description: OpenBSD Secure Shell server ### END INIT INFO set -e # /etc/init.d/ssh: start and stop the OpenBSD "secure shell(tm)" daemon test -x /usr/sbin/sshd || exit 0 umask 022 if test -f /etc/default/ssh; then [...]

This goes on for a total of 162 lines. This is a complete systemd service file from Ubuntu 20.04, /lib/systemd/system/ssh.service:

[Unit] Description=OpenBSD Secure Shell server Documentation=man:sshd(8) man:sshd_config(5) After=network.target auditd.service ConditionPathExists=!/etc/ssh/sshd_not_to_be_run [Service] EnvironmentFile=-/etc/default/ssh ExecStartPre=/usr/sbin/sshd -t ExecStart=/usr/sbin/sshd -D $SSHD_OPTS ExecReload=/usr/sbin/sshd -t ExecReload=/bin/kill -HUP $MAINPID KillMode=process Restart=on-failure RestartPreventExitStatus=255 Type=notify RuntimeDirectory=sshd RuntimeDirectoryMode=0755 [Install] WantedBy=multi-user.target Alias=sshd.service

Even without reading the documentation, or knowing anything about systemd, you can understand some of what this file is supposed to do.

See Rethinking PID 1 for a detailed introduction to systemd by one of its inventors and maintainers, Lennart Poettering. Rethinking PID 1 details the rationale behind building a new init system, its architecture, advantages, and how it uses existing Linux kernel features in place of duplicating existing functionality.

You need to know if your Linux distribution uses systemd or something else.

Look for the /run/systemd/system/ directory. If this exists, then your init system is systemd.

The /run/systemd/ directory may be present on your system if your distribution supports multiple init systems. But systemd is not the active init unless you see /run/systemd/system/.

There are several other ways to learn which init system your system is using. Try querying /sbin/init. Originally this was the SysV executable, and now most Linux distributions preserve the name and symlink it to the systemd executable. This example confirms that the init is systemd:

$ stat /sbin/init File: /sbin/init -> /lib/systemd/systemd [...]

On a system using SysV init, it has no symlink:

$ stat /sbin/init File: /sbin/init [...]

The /proc pseudofilesystem is an interface to your Linux kernel, and contains the current state of a running system. It is called a pseudofilesystem because it exists only in memory and not on disk. In this example, /proc/1/exe is symlinked to the systemd executable:

$ sudo stat /proc/1/exe File: /proc/1/exe -> /lib/systemd/systemd [...]

On a SysV system, it links to init:

$ sudo stat /proc/1/exe File: /proc/1/exe -> /sbin/init [...]

The /proc/1/comm file reports your active init system:

$ cat /proc/1/comm systemd

On a SysV system, it reports init:

$ cat /proc/1/comm init

The command attached to process ID (PID) 1 is your init. PID 1 is the first process launched at startup, which then starts all other processes. You can see this with the ps command:

$ ps -p 1 PID TTY TIME CMD 1 ? 00:00:00 systemd

When the init is SysV, it looks like this:

$ ps -p 1 PID TTY TIME CMD 1 ? 00:00:00 init

See Recipe 4.2 for more information on PID 1.

Linux support for systemd varies. Most of the major Linux distributions have adoped systemd, including Fedora, Red Hat, CentOS, openSUSE, SUSE Linux Enterprise, Debian, Ubuntu, Linux Mint, Arch, Manjaro, Elementary, and Mageia Linux.

Some popular distributions that do not support systemd, or include it but not as the default init, are Slackware, PCLinuxOS, Gentoo Linux, MX Linux, and antiX.

  • Distrowatch for information on hundreds of Linux distributions

  • man 5 proc

  • man 1 pstree

  • man 1 ps

You want a better understanding of services and processes on Linux.

PID 1 is the mother of all processes on Linux systems. This is the first process to start, and then it launches all other processes.

Processes are one or more running instances of a program. Every task in a Linux system is performed by a process. Processes can create independent copies of themselves, that is, they can fork. The forked copies are called children, and the original is the parent. Each child has its own unique PID, and its own allocation of system resources, such as CPU and memory. Threads are lightweight processes that run in parallel and share system resources with their parents.

Some processes run in the background and do not interact with users. Linux calls these processes services or daemons, and their names tend to end with the letter D, such as httpd, sshd, and systemd.

Every Linux system starts PID 1 first, which then launches all other processes. Use the ps command to list all running processes in PID order:

$ ps -ef UID PID PPID C STIME TTY TIME CMD root 1 0 0 10:06 ? 00:00:01 /sbin/init splash root 2 0 0 10:06 ? 00:00:00 [kthreadd] root 3 2 0 10:06 ? 00:00:00 [rcu_gp] root 4 2 0 10:06 ? 00:00:00 [rcu_par_gp] [...]

The pstree command organizes this mass of information into a tree diagram. This example shows all processes, their child processes, PIDs, and threads, which are enclosed in curly braces:

$ pstree -p systemd(1)─┬─ModemManager(925)─┬─{ModemManager}(944) │ └─{ModemManager}(949) ├─NetworkManager(950)─┬─dhclient(1981) │ ├─{NetworkManager}(989) │ └─{NetworkManager}(991) ├─accounts-daemon(927)─┬─{accounts-daemon}(938) │ └─{accounts-daemon}(948) ├─acpid(934) ├─agetty(1103) ├─avahi-daemon(953)───avahi-daemon(970) [...]

The full pstree output is quite large. You can view a single process, identified by its PID, and its parents, children, and threads, like the following example for the Kate text editor:

$ pstree -sp 5193 systemd(1)───kate(5193)─┬─bash(5218) ├─{kate}(5195) ├─{kate}(5196) ├─{kate}(5197) ├─{kate}(5198) ├─{kate}(5199) [...]

This shows that systemd(1) is Kate’s parent, bash(5218) is Kate’s child, and all the processes in curly braces are Kate’s threads.

Processes always exist in one of several states, and these states change according to system activity. The following pstree example displays the PID, user, state, and command fields:

$ ps -eo pid,user,stat,comm PID USER STAT COMMAND 1 root Ss systemd 2 root S kthreadd 32 root I< kworker/3:0H-kb 68 root SN khugepaged 11222 duchess Rl konsole
  • R is either currently running or waiting in the run queue.

  • l means the process is multithreaded.

  • S is interruptable sleep; the process is waiting for an event to complete.

  • s is a session leader. Sessions are related processes managed as a unit.

  • I is an idle kernel thread.

  • < means high priority.

  • N is low priority.

There are several rarely used states you can read about in man 1 ps.

  • Recipe 4.7
  • man 5 proc
  • man 1 pstree
  • man 1 ps

You want to list all services installed on your system, and you want to know the states of the services: whether they are running, not running, or in an error state.

systemctl, the systemd manager command, tells all. Run it with no options to see a detailed list of all loaded units. A systemd unit is any related batch of processes defined in a unit configuration file and managed by systemd:

$ systemctl

This prints a giant pile of information: 177 active loaded units on my test system with the full unit names, status, and long descriptions. Redirect the the output to a text file for easier study:

$ systemctl > /tmp/systemctl-units.txt

Treat yourself to more information overload by listing all units, active and inactive:

$ systemctl --all

This results in 349 loaded units listed on my test system, including not-found and inactive units. How many total unit files? The following example shows 5 out of 322:

$ systemctl list-unit-files UNIT FILE STATE proc-sys-fs-binfmt_misc.automount static -.mount generated mount generated dev-hugepages.mount static home.mount generated [...] 322 unit files listed.

We are interested in service files because Linux users and administrators interact mainly with service files and rarely need to bother with any other type of unit file. How many are installed? Let’s see:

$ systemctl list-unit-files --type=service UNIT FILE STATE accounts-daemon.service enabled acpid.service disabled alsa-state.service static alsa-utils.service masked anacron.service enabled [...] 212 unit files listed.

The preceding example displays the four most common states that a service can be in: enabled, disabled, static, or masked.

List only enabled services:

$ systemctl list-unit-files --type=service --state=enabled UNIT FILE STATE accounts-daemon.service enabled anacron.service enabled apparmor.service enabled [email protected] enabled avahi-daemon.service enabled [...] 62 unit files listed.

List only disabled services:

$ systemctl list-unit-files --type=service --state=disabled UNIT FILE STATE acpid.service disabled brltty.service disabled console-getty.service disabled [email protected] disabled [...] 12 unit files listed.

List only static services:

$ systemctl list-unit-files --type=service --state=static UNIT FILE STATE alsa-restore.service static alsa-state.service static apt-daily-upgrade.service static apt-daily.service static [...] 106 unit files listed.

List only masked services:

$ systemctl list-unit-files --type=service --state=masked UNIT FILE STATE alsa-utils.service masked bootlogd.service masked bootlogs.service masked checkfs.service masked [...] 36 unit files listed.

Service unit files are in /usr/lib/systemd/system/ or /lib/systemd/system/, according to where your Linux distribution puts them. These are plain-text files you can read.

enabled

This shows that the service has become available and is managed by systemd. When a service is enabled, systemd creates a symlink in /etc/systemd/system/ from the unit file in /lib/systemd/system/. It can be started, stopped, reloaded, and disabled by the user with the systemctl command.

Enabling a service does not immediately start it, and disabling a service does not immediately stop it (see Recipe 4.6).

disabled

Diabled means that there is no symlink in /etc/systemd/system/, and it will not start automatically at boot. You can stop and start it manually.

masked

This means the service is linked to /dev/null/. It is completely disabled and cannot be started by any means.

static

This means that the unit file is a dependency of other unit files, and cannot be started or stopped by the user.

Some less-common service states you will see:

indirect

Indirect states belong to services that are not meant to be managed by users, but are meant to be used by other services.

generated

Generated states indicate that the service has been converted from a nonnative systemd initialization configuration file, either SysV or LSB init.

You want to know the status of one service or a few specific services.

systemctl status provides a nice little bundle of useful status information. The following example queries the CUPS service. CUPS, the Common Unix Printing System, should be on all Linux systems:

$ systemctl status cups.service ● cups.service - CUPS Scheduler Loaded: loaded (/lib/systemd/system/cups.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2021-11-22 11:01:48 PST; 4h 17min ago TriggeredBy: ● cups.path ● cups.socket Docs: man:cupsd(8) Main PID: 1403 (cupsd) Tasks: 2 (limit: 18760) Memory: 3.8M CGroup: /system.slice/cups.service ├─1403 /usr/sbin/cupsd -l └─1421 /usr/lib/cups/notifier/dbus dbus:// Nov 22 11:01:48 host1 systemd[1]: Started CUPS Scheduler.

Query multiple services with a space-delimited list:

$ systemctl status mariadb.service bluetooth.service lm-sensors.service

There is a lot of useful information in this little bit of output (Figure 4-2).

What are the two system initialization processes utilized by linux to load and manage services?

The dot next to the service name is a quick status indicator. It appears in colors on most terminals. White is an inactive or deactivating state. Red is a failed or error state. Green indicates an active, reloading, or activating state. The rest of the information in the output is described in the following:

Loaded

Verifies that the unit file has been loaded into memory, displays its full path, the service is enabled (see the Discussion about states in Recipe 4.3), and vendor preset: disabled/enabled indicates if the installation default is to start at boot or not. When it is disabled, the vendor default is to not start at boot. This only shows the vendor preference and does not indicate if it is currently enabled or disabled.

Active

Tells you if the service is active or inactive, and how long it has been in that state.

Process

Reports the PIDs and their commands and daemons.

Main PID

This is the process number for the cgroup slice.

Tasks

Reports how many tasks the service has started. Tasks are PIDs.

CGroup

Shows which unit slice the service belongs to and its PID. The three default unit slices are user.slice, system.slice, and machine.slice.

Linux control groups (cgroups) are sets of related processes and all of their future children. In systemd, a slice is a subdivision of a cgroup, and each slice manages a particular group of processes. Run systemctl status to see a diagram of the cgroup hierarchy.

By default, service and scope units are grouped in /lib/systemd/system/system.slice.

User sessions are grouped in /lib/systemd/system/user.slice.

Virtual machines and containers registered with systemd are grouped in /lib/systemd/system/machine.slice.

The remaining lines are the most recent log entries from journalctl, the systemd log manager.

You want to stop and start services with systemd.

This is a job for systemctl. The following examples use the SSH service to demonstrate service management.

Start a service:

$ sudo systemctl start sshd.service

Stop a service:

$ sudo systemctl stop sshd.service

Stop and then restart a service:

$ sudo systemctl restart sshd.service

Reload the service’s configuration. For example, you made a change to sshd_config and want to load the new configuration without restarting the service:

$ sudo systemctl reload sshd.service

All of these commands also work with multiple services, space-delimited, for example:

$ sudo systemctl start sshd.service mariadb.service firewalld.service

If you’re curious about the commands that systemd runs behind the scenes to start, reload, or stop the individual daemons, look in their unit files. Some services have start, reload, stop, and other instructions in their unit files, like this example for httpd:

ExecStart=/usr/sbin/httpd/ $OPTIONS -DFOREGROUND ExecReload=/usr/sbin/httpd $OPTIONS -k graceful ExecStop=/bin/kill -WINCH ${MAINPID}

You don’t have to do anything special with this information; it is there when you want to know how systemctl is managing a particular service.

  • Recipe 4.6

  • man 1 systemctl

You want a service or services to automatically start at boot, or you want to prevent a service from starting at boot, or to disable it completely.

Enabling a service configures it to automatically start at boot.

Disabling a service stops it from starting at boot, but it can be started and stopped manually.

Masking a service disables it so that it cannot be started at all.

The following example enables the sshd service:

$ sudo systemctl enable sshd.service Created symlink /etc/systemd/system/multi-user.target.wants/sshd.service → /usr/lib/systemd/system/sshd.service

The output shows that enabling a service means creating a symlink from the service file in /lib/systemd/system/ to /etc/systemd/system/. This does not start the service. You can start the service with systemctl start, or enable and start the service in one command with the --now option:

$ sudo systemctl enable --now sshd.service

This command disables the sshd service. It does not stop the service, so you must stop it manually after disabling it:

$ sudo systemctl disable sshd.service Removed /etc/systemd/system/multi-user.target.wants/sshd.service $ sudo systemctl stop sshd.service

Or, disable and stop it with one command:

$ sudo systemctl disable --now sshd.service

This command reenables the mariadb service, which disables and then enables it. If you have created the symlinks manually for a service, this is useful for quickly resetting them to the defaults:

$ sudo systemctl reenable mariadb.service Removed /etc/systemd/system/multi-user.target.wants/mariadb.service. Removed /etc/systemd/system/mysqld.service. Removed /etc/systemd/system/mysql.service. Created symlink /etc/systemd/system/mysql.service → /lib/systemd/system/mariadb.service. Created symlink /etc/systemd/system/mysqld.service → /lib/systemd/system/mariadb.service. Created symlink /etc/systemd/system/multi-user.target.wants/mariadb.service → /lib/systemd/system/mariadb.service.

The following command disables the bluetooth service completely by masking it, so that it cannot be started at all:

$ sudo systemctl mask bluetooth.service Created symlink /etc/systemd/system/bluetooth.service → /dev/null.

Unmasking the bluetooth service does not enable it, so it must be started manually:

$ sudo systemctl unmask bluetooth.service Removed /etc/systemd/system/bluetooth.service. $ sudo systemctl start bluetooth.service

When you enable, disable, mask, or unmask a service, it remains in its current state unless you use the --now option. The --now option works with enable, disable, and mask to immediately stop or start the service, but it does not work with unmask.

See the Discussion in Recipe 4.3 to learn more about how systemd uses symlinks to manage services.

  • man 1 systemctl

  • The Discussion in Recipe 4.3 to learn how systemd uses symlinks to manage services

You want to know how to stop troublesome processes. A certain service may be unresponsive or running away, spawning forks and causing your system to hang. Your normal stop command is not working. What do you do?

Stopping a process is called killing the process. On Linux systems with systemd, you should use systemctl kill. On systems without systemd, use the legacy kill command.

systemctl kill is preferable because it stops all processes that belong to a service and leaves no orphan processes, nor any processes that might restart the service and continue to make trouble. First, try it with no options other than the service name, then check the status:

$ sudo systemctl kill mariadb $ systemctl status mariadb ● mariadb.service - MariaDB 10.1.44 database server Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled) Active: inactive (dead) since Sun 2020-06-28 19:57:49 PDT; 6s ago [...]

The service has cleanly stopped. If this does not work, then try the nuclear option:

$ sudo systemctl kill -9 mariadb

The legacy kill command does not recognize service or command names, but rather requires the PID of the offending process:

$ sudo kill 1234

If this does not stop it, use the nuclear option:

$ sudo kill -9 1234

Use the top command to identify runaway processes. Run it with no options, and the processes using up the most CPU resources are listed at the top. Press the q key to exit top.

$ top top - 20:30:13 up 4:24, 6 users, load average: 0.00, 0.03, 0.06 Tasks: 246 total, 1 running, 170 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.4 us, 0.2 sy, 0.0 ni, 99.4 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem : 16071016 total, 7295284 free, 1911276 used, 6864456 buff/cache KiB Swap: 8928604 total, 8928604 free, 0 used. 13505600 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3504 madmax 20 0 99.844g 177588 88712 S 2.6 1.1 0:08.68 evolution 2081 madmax 20 0 3818636 517756 177744 S 0.7 3.2 5:07.56 firefox 1064 root 20 0 567244 148432 125572 S 0.3 0.9 12:54.75 Xorg 2362 stash 20 0 2997732 230508 145444 S 0.3 1.4 0:40.72 Web Content [...]

kill sends signals to processes, and the default signal is SIGTERM (signal terminate). SIGTERM is gentle, allowing processes to shut down cleanly. SIGTERM is also ignorable, and processes don’t have to pay attention to it. Signals can be identified by name or number; for most folks the numbers are easier to remember, so spelling out the default looks like this:

$ sudo kill -1 1234

kill -9 is SIGKILL. SIGKILL stops processes immediately and uncleanly, and also attempts to stop all child processes.

Killing services with systemctl kill is easier than with kill, and more reliable. You only need the service name, and you don’t have to hunt down PIDs. It ensures that all processes belonging to the service are stopped, which kill cannot ensure.

There are a ton of signals that have accumulated over the years, and you can read all about them in man 7 signal. In my experience, the most relevant signals are SIGTERM and SIGKILL, but don’t let that stop you from learning more about the others.

If you are uncomfortable with terminology like kill, parents, children, and orphans, so am I. Maybe someday it will change.

  • man 5 systemd.kill

  • man 1 systemctl

  • man 1 kill

  • man 7 signal

You want to reboot to different system states in a manner similar to using SysV runlevels.

systemd targets are similar to SysV runlevels. These are boot profiles that start your system with different options, such as multiuser mode with a graphical desktop, multiuser mode with no graphical desktop, and emergency and rescue modes to use when your current target will not boot. (See the Discussion for more information on runlevels.)

The following command checks if the system is running and reports its state:

$ systemctl is-system-running running

What is the default target?

$ systemctl get-default graphical.target

Get the current runlevel:

$ runlevel N 5

Reboot to rescue mode:

$ sudo systemctl rescue

Reboot to emergency mode:

$ sudo systemctl emergency

Reboot to the default mode:

$ sudo systemctl reboot

Reboot to a different target without changing the default:

$ sudo systemctl isolate multi-user.target

Set a different default runlevel:

$ sudo systemctl set-default multi-user.target

List the runlevel target files and their symlinks on your system:

$ ls -l /lib/systemd/system/runlevel*

List the dependencies in a runlevel target:

$ systemctl list-dependencies graphical.target

SysV runlevels are different states that your system can boot to, for example, with a graphical desktop, without a graphical desktop, and with emergency runlevels to use when your default runlevel has problems and will not boot.

systemd targets approximately correspond to the legacy SysV runlevels:

  • runlevel0.target, poweroff.target, halt

  • runlevel1.target, rescue.target, single-user text mode, all local filesystems mounted, root user only, no networking

  • runlevel3.target, multi-user.target, multiuser text mode (no graphical environment)

  • runlevel5.target, graphical.target, multiuser graphical mode

  • runlevel6.target, reboot.target, reboot

systemctl emergency is a special target that is more restricted than rescue mode: no services, no mount points other than the root filesystem, no networking, root user only. It is the most minimal running system for debugging problems. You may see options to boot into a rescue or emergency mode in your GRUB2 bootloader screen.

systemctl is-system-running reports various system states:

  • initializing means the system has not completed startup.

  • starting means the system is in the final stages of startup.

  • running is fully operational, and all processes are started.

  • degraded means the system is operational, but one or more systemd units have failed. Run systemctl | grep failed to see which units failed.

  • maintenance means that either the rescue or emergency target is active.

  • stopping means that systemd is shutting down.

  • offline means that systemd is not running.

  • unknown means that there is a problem preventing systemd from determining the operational state.

  • man 1 systemctl

  • man 8 systemd-halt.service

systemd promises faster startups, but your system starts up slowly, and you want to find out why.

You want systemd-analyze blame. Run it with no options to see a list of system processes and how long they took to start:

$ systemd-analyze blame 34.590s apt-daily.service 6.782s NetworkManager-wait-online.service 6.181s dev-sda2.device 4.444s systemd-journal-flush.service 3.609s udisks2.service 2.450s snapd.service [...]

Analyze only user processes:

$ systemd-analyze blame --user 3.991s pulseaudio.service 553ms at-spi-dbus-bus.service 380ms evolution-calendar-factory.service 331ms evolution-addressbook-factory.service 280ms xfce4-notifyd.service [...]

It is useful to review everything that starts at boot and perhaps find services you don’t want starting at boot. My favorite to disable is Bluetooth because I don’t use it on my servers or PCs, but many Linux distros enable it by default.

Get Linux Cookbook, 2nd Edition now with the O’Reilly learning platform.

O’Reilly members experience live online training, plus books, videos, and digital content from nearly 200 publishers.