When using the new RHEL 7.2 release and running the systemctl status command, some slight changes were recently visible:

# systemctl status sshd
● sshd.service - OpenSSH server daemon
Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2016-02-13 10:56:40 CET; 6s ago
Docs: man:sshd(8)
man:sshd_config(5)
Main PID: 1286 (sshd)
CGroup: /system.slice/sshd.service
└─1286 /usr/sbin/sshd -D

Feb 13 10:56:40 server.example.com systemd[1]: Started OpenSSH server daemon.
Feb 13 10:56:40 server.example.com systemd[1]: Starting OpenSSH server daem....
Feb 13 10:56:40 server.example.com sshd[1286]: Server listening on 0.0.0.0 ....
Feb 13 10:56:40 server.example.com sshd[1286]: Server listening on :: port 22.
Feb 13 10:56:44 server.example.com sshd[2457]: Accepted password for root f...2
Hint: Some lines were ellipsized, use -l to show in full.

As you can see above, a new string vendor preset: enabled is now displayed. Here it means that the preset mechanism has been applied on the sshd package.

The preset mechanism came with Systemd and the RHEL 7.0 release, the RHEL 7.2 release just makes it visible. It is a way for a Linux distribution (Debian, CentOS, etc) to centrally define whether a package should be automatically enabled or not at installation time.

So, through the well known directory hierarchy (/usr/lib/systemd/system-preset, /etc/systemd/system-preset, /run/systemd/system-preset), a Linux distribution can specify how packages initially behave.

A typical preset file would appear like this:

# more /usr/lib/systemd/system-preset/90-default.preset
# System stuff
enable sshd.service
enable atd.*
enable crond.*
enable chronyd.service
enable rpcbind.*
enable NetworkManager.service
enable NetworkManager-dispatcher.service
enable ModemManager.service
enable auditd.service
enable restorecond.service
enable bluetooth.*
enable avahi-daemon.*
enable cups.*

# The various syslog implementations
enable rsyslog.*
enable syslog-ng.*
enable sysklogd.*

# Network facing
enable firewalld.service
enable libvirtd.service
enable xinetd.service
enable ladvd.service
...

A directive enable or disable precedes each package in this file as appropriate.

You shouldn’t need to modify this file in most cases but it is still interesting to know that it exists.

Sources: systemd.preset man page and Freedesktop.org website.

Among the changes coming with the RHEL 7.2 release, there is the new systemctl edit command.

Like the systemctl cat command, you specify a service unit file as argument. But, unlike the systemctl cat command, you will need to think twice before using it in a production environment. All successful editing converts into a service restart!

So, without any other argument than the unit file, you create or edit a drop-in snippet called override.conf that will complement the current service configuration. If necessary the associated service directory is created (for example /etc/systemd/system/httpd.service.d in the httpd case) which is pretty handy.

You need to set up one of the following environment variables before: SYSTEMD_EDITOR, EDITOR or VISUAL. If you don’t do it, the nano, vim, and vi editors will be tried one after the other.

If you add the –full option, it is now the service unit file that you are editing and not a drop-in snippet anymore.

Finally, you can use the –runtime option, if you want to make some changes to a service unit file in the /usr/lib/systemd/system directory and don’t want the change to survive a reboot (this doesn’t work if the unit file is located in the /etc/systemd/system directory because of precedence between /etc and /run).

This command is definitively a useful addition for speeding up the tuning of service unit files in a development environment.

In a previous post, I presented the various mechanisms used for customizing Systemd unit files.
These mechanisms are so rich that it is not easy to get the real picture of the final unit file.
You have to search in different locations (/usr/lib/systemd/system, /etc/systemd/system, /etc/systemd/system/myservice.service.d/).

Coming with the RHEL 7.2 release, the new systemctl cat command serves two main purposes:

• to allow an administrator to quickly display the content of a given unit file,
• to take into account drop-in files when debugging unit files.

For example, let’s install the mariadb-server package:

# yum install -y mariadb-server

Now, let’s create the /etc/systemd/system/mariadb.service.d directory:

# mkdir /etc/systemd/system/mariadb.service.d
# cd /etc/systemd/system/mariadb.service.d

Then, let’s create the foo.conf file and paste the following lines into it:

[Service]
LimitNOFILE=10000

Finally, let’s display the mariadb unit file:

# systemctl cat mariadb.service
# /usr/lib/systemd/system/mariadb.service <== unit file location
[Unit]
Description=MariaDB database server
After=syslog.target
After=network.target

[Service]
Type=simple
User=mysql
Group=mysql

ExecStartPre=/usr/libexec/mariadb-prepare-db-dir %n
ExecStart=/usr/bin/mysqld_safe --basedir=/usr
ExecStartPost=/usr/libexec/mariadb-wait-ready $MAINPID

# Give a reasonable amount of time for the server to start up/shut down
TimeoutSec=300

# Place temp files in a secure directory, not /tmp
PrivateTmp=true

[Install]
WantedBy=multi-user.target

# /etc/systemd/system/mariadb.service.d/foo.conf <== drop-in file
[Service]
LimitNOFILE=10000

Now, you can see the unit file location displayed at the top and the content of the drop-in file at the end.

The systemctl cat command should become a reflex for all the system administrators when they want to know the precise configuration of a given service.

Note: Most of the original comments have been removed from the mariadb.service file here for brevity’s sake. I invit you to look at it in details because it provides a lot of useful comments, reminding you of the available Systemd customization mechanisms.

From my understanding, Systemd was created to make the maintenance of Linux distributions easier.

With previous init systems, you had to know all packages that could interact with a given subsystem, and you had to know them for each distribution for the same subsystem.

For subsystems as complicated as NFS, it was certainly a real pain.

At the same time, Systemd came with added features solving several problems of previous init systems:

• dynamic dependencies management,
• parallel execution of initial processes making boot faster,
• xinetd replacement through socket-activation,
• respawn of services as needed,
• declarative configuration files instead of complicated scripts,
• powerful customization of service configuration files,
• etc.

Concerning Unix philosophy, gurus could discuss “Do only one thing but do it well” principle during hours against Systemd integration model, this would not change anything.

So, all was pretty good except one point: to know how to administer a complicated subsystem, you now had to look at the /usr/lib/systemd/system directory, grep on keywords and guess which unit files needed to be started and enabled at boot! This could become very tricky.

Some problems started with the iSCSI subsystem where a service called target.service came out(what a strange and confusing name!). This unit file didn’t even come from one of the two main iSCSI packages (targetcli and iscsi-initiator-utils) but from a dependency (python-rtslib)! To understand that you had to enable it to get your iSCSI target configuration saved took some time.

Then came some NFS interface changes with RHEL 7.1. Some service configuration files were renamed, some disappeared, others were created(*). What a mess! And no explanation at all! You had to be a NFS guru and keep reading the NFS mailing list to understand what was going on.

Now, you must ask yourself several questions:

• Why the public Red Hat documentation doesn’t yet advertise the main services and targets required when administering a subsystem like NFS? Search for nfs-client.target on the Red Hat website to get your own idea.
• Is it really the Open Source spirit? 
• Besides the .service, .target, .socket, why there isn’t a .info file describing how to administer a given subsystem?

Systemd has made maintenance of Linux distributions easier, why wouldn’t it make life of administrators easier now?
 
(*)The first command to type when taking the RHCE 7 exam is now:

# cat /etc/redhat-release

Yesterday, in the CentOS mailing list, Karanbir Singh, the CentOS project lead, announced the CentOS 7.2 release. Details are available in the CentOS 7 (1511) Releases Notes.

Also, the Docker container, Vagrant images, Cloud images and Atomic Host images will be released in the next few days with ARMv7 (armhfp), Aarch64 and i686 architectures coming slightly later. Furthermore, several SIGs (Special Interest Group) like Cloud SIG, Cloud Instance SIG, Virtualization SIG, Storage SIG, Software Collections SIG and Atomic SIG will release specific versions.

CentOS 7.2 globally brings the same main changes as RHEL 7.2: Systemd v219, Gnome 3.14, OpenJDK with ECC support, etc.

Finally, one line caught my attention in the CentOS 7 (1511) Release Notes:

“The initramfs files are now significantly bigger than in CentOS-7 (1503). You may want to consider lowering installonly_limit in /etc/yum.conf to reduce the number of installed kernels if your /boot partition is smaller than 4GB. New installations should consider using 1GB as the size of the /boot partition.”

This is something to think about, particularly in virtualized environments where VM size matters.
Otherwise, as Murphy‘s law says, if anything can go wrong, it will!

The website is mainly about RHEL 7/CentOS 7 but the distribution running the webserver was still CentOS 6: something was wrong.
This week I have been busy migrating from a physical server on CentOS 6 to a much powerful virtual machine on CentOS 7.
I hope you will find the website more responsive.

Otherwise, we have seen Microsoft and Red Hat signing several agreements recently, something that most people thought they would never see. Even better, Microsoft started a new certification program around Linux on Azure!
Unbelievable!

Recently on Reddit, someone wrote:

[…]There are minor differences in command syntax when comparing CentOS and RHEL. For example, nmcli and the lvm management both functioned incorrectly I found when I was using the utilities extensively on CentOS (this is compared to what was advised in the RHEL course, and alongside my past knowledge of LVM, the errors being presented shouldn’t of occurred – and indeed didn’t when I tried repeated them on RHEL the following day).

[…]Perfect example is the following: You’ve got an existing connection named “System eth0”, and you plan to adjust the “ipv4.addresses” field to set an IP address and a gateway. In RHEL, you’d type the following:
nmcli connection modify “System eth0” ipv4.addresses “10.0.0.1/24 10.0.0.254”
This would set the first field as the IP, and the second as the gateway.

In CentOS however, you’ll get an error like the following:
“ERROR: 24 10.0.0.254 isn’t a valid prefix.”

Because many candidates intensively use CentOS for preparing their exams, I thought this question would deserve a clear answer. I asked Karanbir Singh, the lead developer for CentOS, via Twitter.
Here, what he answered:

that’s an incorrect assertion, NetworkManager / nmcli has been verified to have similar functionality.

Fabian Arrotin, a member of the CentOS team, also wrote:

difference between NetworkManager 0.9.9 from el 7.0 and then bumped to 1.0.0 in el7.1 ? (and RHCE course for 7.0) ?

I found Fabian‘s answer very interesting.

I launched two VMs, one in RHEL 7.0 and one in RHEL 7.1. I then checked the behaviour of the nmcli command. Here what I got:

On the RHEL 7.0 VM:

# nmcli con mod "System eth0" ipv4.addresses "192.168.122.80/24 192.168.122.254"
#

On the RHEL 7.1 VM:

# nmcli con mod "System eth0" ipv4.addresses "192.168.122.80/24 192.168.122.254"
Error: failed to modify ipv4.addresses: invalid prefix '24 192.168.122.254'; <1-32> allowed.

How could you say that CentOS doesn’t behave like RHEL when two versions of RHEL give different results? This is the version of NetworkManager that matters!

In the RHEL 7.0 case, you’ve got NetworkManager-0.9.9.1.
In the RHEL 7.1 case, it is NetworkManager-1.0.0.

I found one bug fix affecting the gateway syntax (rh #1170199: Unable to add static IP address with gateway) in the changelog between the two versions but there are perhaps more of them.

Conclusion1: Don’t use this kind of syntax any more:

# nmcli con mod "System eth0" ipv4.addresses "10.0.0.1/24 10.0.0.254"

Do it in two steps:

# nmcli con mod "System eth0" ipv4.addresses 10.0.0.1/24
# nmcli con mod "System eth0" ipv4.gateway 10.0.0.254

Conclusion2: It is rather easy to spread a wrong rumor, but this is no good for anyone!