When preparing an exam like the RHCSA or the RHCE exams, it is relatively difficult to remember the commands with all their options, the configuration procedures with all their details, and finally the mistakes to avoid. This is even worse if you have little or no real experience. In this case, you have to make up for your lack of experience by being clever.

One of the main tips is to know how to find the relevant information in the current system itself through man pages, default configuration files with theirs comments (Apache, Samba, etc), and configuration examples coming with some packages (Postfix, MariaDB, etc).

Recently one reader, Carlos G Mendioroz, left a very good comment about teaming configuration. If you don’t remember some details, several configurations are available in the /usr/share/doc/teamd-*/example_configs directory.

For example, to get an example of a round robin configuration, type:

# more /usr/share/doc/teamd-1.9/example_configs/roundrobin.conf
{
"device": "team0",
"runner": {"name": "roundrobin"},
"ports": {"eth1": {}, "eth2": {}}
}

It is clear that this doesn’t explain everything nor replace practice. If you spend your time searching for man pages information or searching for example configuration, time will run out and you won’t be able to complete the different tasks in time.

When using the new RHEL 7.2 release and running the systemctl status command, some slight changes were recently visible:

# systemctl status sshd
● sshd.service - OpenSSH server daemon
Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2016-02-13 10:56:40 CET; 6s ago
Docs: man:sshd(8)
man:sshd_config(5)
Main PID: 1286 (sshd)
CGroup: /system.slice/sshd.service
└─1286 /usr/sbin/sshd -D

Feb 13 10:56:40 server.example.com systemd[1]: Started OpenSSH server daemon.
Feb 13 10:56:40 server.example.com systemd[1]: Starting OpenSSH server daem....
Feb 13 10:56:40 server.example.com sshd[1286]: Server listening on 0.0.0.0 ....
Feb 13 10:56:40 server.example.com sshd[1286]: Server listening on :: port 22.
Feb 13 10:56:44 server.example.com sshd[2457]: Accepted password for root f...2
Hint: Some lines were ellipsized, use -l to show in full.

As you can see above, a new string vendor preset: enabled is now displayed. Here it means that the preset mechanism has been applied on the sshd package.

The preset mechanism came with Systemd and the RHEL 7.0 release, the RHEL 7.2 release just makes it visible. It is a way for a Linux distribution (Debian, CentOS, etc) to centrally define whether a package should be automatically enabled or not at installation time.

So, through the well known directory hierarchy (/usr/lib/systemd/system-preset, /etc/systemd/system-preset, /run/systemd/system-preset), a Linux distribution can specify how packages initially behave.

A typical preset file would appear like this:

# more /usr/lib/systemd/system-preset/90-default.preset
# System stuff
enable sshd.service
enable atd.*
enable crond.*
enable chronyd.service
enable rpcbind.*
enable NetworkManager.service
enable NetworkManager-dispatcher.service
enable ModemManager.service
enable auditd.service
enable restorecond.service
enable bluetooth.*
enable avahi-daemon.*
enable cups.*

# The various syslog implementations
enable rsyslog.*
enable syslog-ng.*
enable sysklogd.*

# Network facing
enable firewalld.service
enable libvirtd.service
enable xinetd.service
enable ladvd.service
...

A directive enable or disable precedes each package in this file as appropriate.

You shouldn’t need to modify this file in most cases but it is still interesting to know that it exists.

Sources: systemd.preset man page and Freedesktop.org website.

Among the changes coming with the RHEL 7.2 release, there is the new systemctl edit command.

Like the systemctl cat command, you specify a service unit file as argument. But, unlike the systemctl cat command, you will need to think twice before using it in a production environment. All successful editing converts into a service restart!

So, without any other argument than the unit file, you create or edit a drop-in snippet called override.conf that will complement the current service configuration. If necessary the associated service directory is created (for example /etc/systemd/system/httpd.service.d in the httpd case) which is pretty handy.

You need to set up one of the following environment variables before: SYSTEMD_EDITOR, EDITOR or VISUAL. If you don’t do it, the nano, vim, and vi editors will be tried one after the other.

If you add the –full option, it is now the service unit file that you are editing and not a drop-in snippet anymore.

Finally, you can use the –runtime option, if you want to make some changes to a service unit file in the /usr/lib/systemd/system directory and don’t want the change to survive a reboot (this doesn’t work if the unit file is located in the /etc/systemd/system directory because of precedence between /etc and /run).

This command is definitively a useful addition for speeding up the tuning of service unit files in a development environment.

In a previous post, I presented the various mechanisms used for customizing Systemd unit files.
These mechanisms are so rich that it is not easy to get the real picture of the final unit file.
You have to search in different locations (/usr/lib/systemd/system, /etc/systemd/system, /etc/systemd/system/myservice.service.d/).

Coming with the RHEL 7.2 release, the new systemctl cat command serves two main purposes:

• to allow an administrator to quickly display the content of a given unit file,
• to take into account drop-in files when debugging unit files.

For example, let’s install the mariadb-server package:

# yum install -y mariadb-server

Now, let’s create the /etc/systemd/system/mariadb.service.d directory:

# mkdir /etc/systemd/system/mariadb.service.d
# cd /etc/systemd/system/mariadb.service.d

Then, let’s create the foo.conf file and paste the following lines into it:

[Service]
LimitNOFILE=10000

Finally, let’s display the mariadb unit file:

# systemctl cat mariadb.service
# /usr/lib/systemd/system/mariadb.service <== unit file location
[Unit]
Description=MariaDB database server
After=syslog.target
After=network.target

[Service]
Type=simple
User=mysql
Group=mysql

ExecStartPre=/usr/libexec/mariadb-prepare-db-dir %n
ExecStart=/usr/bin/mysqld_safe --basedir=/usr
ExecStartPost=/usr/libexec/mariadb-wait-ready $MAINPID

# Give a reasonable amount of time for the server to start up/shut down
TimeoutSec=300

# Place temp files in a secure directory, not /tmp
PrivateTmp=true

[Install]
WantedBy=multi-user.target

# /etc/systemd/system/mariadb.service.d/foo.conf <== drop-in file
[Service]
LimitNOFILE=10000

Now, you can see the unit file location displayed at the top and the content of the drop-in file at the end.

The systemctl cat command should become a reflex for all the system administrators when they want to know the precise configuration of a given service.

Note: Most of the original comments have been removed from the mariadb.service file here for brevity’s sake. I invit you to look at it in details because it provides a lot of useful comments, reminding you of the available Systemd customization mechanisms.

From my understanding, Systemd was created to make the maintenance of Linux distributions easier.

With previous init systems, you had to know all packages that could interact with a given subsystem, and you had to know them for each distribution for the same subsystem.

For subsystems as complicated as NFS, it was certainly a real pain.

At the same time, Systemd came with added features solving several problems of previous init systems:

• dynamic dependencies management,
• parallel execution of initial processes making boot faster,
• xinetd replacement through socket-activation,
• respawn of services as needed,
• declarative configuration files instead of complicated scripts,
• powerful customization of service configuration files,
• etc.

Concerning Unix philosophy, gurus could discuss “Do only one thing but do it well” principle during hours against Systemd integration model, this would not change anything.

So, all was pretty good except one point: to know how to administer a complicated subsystem, you now had to look at the /usr/lib/systemd/system directory, grep on keywords and guess which unit files needed to be started and enabled at boot! This could become very tricky.

Some problems started with the iSCSI subsystem where a service called target.service came out(what a strange and confusing name!). This unit file didn’t even come from one of the two main iSCSI packages (targetcli and iscsi-initiator-utils) but from a dependency (python-rtslib)! To understand that you had to enable it to get your iSCSI target configuration saved took some time.

Then came some NFS interface changes with RHEL 7.1. Some service configuration files were renamed, some disappeared, others were created(*). What a mess! And no explanation at all! You had to be a NFS guru and keep reading the NFS mailing list to understand what was going on.

Now, you must ask yourself several questions:

• Why the public Red Hat documentation doesn’t yet advertise the main services and targets required when administering a subsystem like NFS? Search for nfs-client.target on the Red Hat website to get your own idea.
• Is it really the Open Source spirit? 
• Besides the .service, .target, .socket, why there isn’t a .info file describing how to administer a given subsystem?

Systemd has made maintenance of Linux distributions easier, why wouldn’t it make life of administrators easier now?
 
(*)The first command to type when taking the RHCE 7 exam is now:

# cat /etc/redhat-release

Yesterday, in the CentOS mailing list, Karanbir Singh, the CentOS project lead, announced the CentOS 7.2 release. Details are available in the CentOS 7 (1511) Releases Notes.

Also, the Docker container, Vagrant images, Cloud images and Atomic Host images will be released in the next few days with ARMv7 (armhfp), Aarch64 and i686 architectures coming slightly later. Furthermore, several SIGs (Special Interest Group) like Cloud SIG, Cloud Instance SIG, Virtualization SIG, Storage SIG, Software Collections SIG and Atomic SIG will release specific versions.

CentOS 7.2 globally brings the same main changes as RHEL 7.2: Systemd v219, Gnome 3.14, OpenJDK with ECC support, etc.

Finally, one line caught my attention in the CentOS 7 (1511) Release Notes:

“The initramfs files are now significantly bigger than in CentOS-7 (1503). You may want to consider lowering installonly_limit in /etc/yum.conf to reduce the number of installed kernels if your /boot partition is smaller than 4GB. New installations should consider using 1GB as the size of the /boot partition.”

This is something to think about, particularly in virtualized environments where VM size matters.
Otherwise, as Murphy‘s law says, if anything can go wrong, it will!