With the upcoming RHEL 7.2 version, Systemd should be upgraded, going from version 208 to 219.

This Systemd upgrade decision was taken according to Lukáš Nykrýn from Red Hat because “it is really difficult and risky to backport patches from newer version to our current 208 and we have a lot of request for new features from users”.

Today, you’ve got two solutions to experiment this new version:

• either install the Fedora 22 distribution on a PC or a virtual machine after downloading the 64-Bit 2.1GB installation image,
• or install a RHEL 7/CentOS 7 distribution, configure the EPEL repository, follow Lukáš Nykrýn’s instructions and update the distribution:

  # yum install -y epel-release
  # cd /etc/yum.repos.d
  # wget "https://copr.fedoraproject.org/coprs/lnykryn/systemd/repo/epel-7/lnykryn-systemd-epel-7.repo"
  # yum -y update

You can now check the new Systemd version:

# systemctl --version
systemd 219
+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN

According to the 2015 Red Hat Summit Systemd Presentation, this version brings the following new features:

• systemctl: new edit and cat options
• CPUQuota: “cap” CPU usage for services
• systemd-socket-proxy: add socket activation to daemons that don’t support it natively
• systemd-nspawn: improvements on networking, container management, etc
• systemd-networkd: DHCP server/client, bridge, bond, vlan, vxlan, macvlan, tun configuration management

Happy testing!

Here are some of the main 2015 Red Hat Summit announcements:

• the upcoming RHEL 7.2 release will bring
  • SCAP profiles into the installation process,
  • cryptography enhancements,
  • kernel patching,
  • performance gains (network and system),
  • a new predictable cycle for software updates,
  • better OS modularity (physical server, virtual machine, cloud, etc),
  • high-availability support for IBM Z-system mainframes,
  • full ARM support.
• the new Red Hat Learning Subscription provides the following benefits:
  • unlimited access to all of Red Hat’s online courses (content, videos, and labs),
  • access to expert seminars given by Red Hat,
  • access to subscriber support with live online chat,
  • 50% discount for current RHCAs,
• the new Red Hat Atomic Enterprise Platform extends the already known Project Atomic offering:
  • a managed cluster of RHEL 7 or Red Hat Enterprise Linux Atomic Host instances,
  • the Docker container runtime and packaging format,
  • container orchestration with Kubernetes,
  • cluster-wide infrastructure services,
  • a commitment to container security and certification,
• the new Openshift Enterprise 3.0 now relies on:
  • Kubernetes,
  • Docker,
  • RHEL 7/Red Hat Atomic Enterprise Platform,
• Red Hat and Samsung work together to deliver the Red Hat Mobile Application Platform, using the MBAAS (Mobile Backend As A Service).

Summit Breakout Sessions are available here. Some presentation slides are there.

The other day I was looking at some Systemd service properties with the systemctl show command. One of them, CPUShares, got my interest.

Caution: The following tutorial shouldn’t be run on a production server! The CPU will be used at 100%!

To better understand how all this works, I created a basic Systemd unit file called /etc/systemd/system/testSpeed.service:

[Unit]
Description=Test Speed
After=syslog.target

[Service]
ExecStart=/usr/bin/openssl speed 
ExecStop=/bin/kill -WINCH ${MAINPID}

[Install]
WantedBy=multi-user.target

Then, I created another copy of this file and updated the Systemd configuration:

# cd /etc/systemd/system; cp testSpeed.service testSpeed2.service
# systemctl daemon-reload

I started both new services on a fresh standard install of Centos 7 on a VM with 1 vCPU:

# systemctl start testSpeed && systemctl start testSpeed2

Each of the two new services were using almost 50% of the CPU:

  PID USER      PR  NI S %CPU %MEM     TIME+ COMMAND      
24598 root      20   0 R 49.8  0.3   0:08.42 openssl      
24601 root      20   0 R 49.8  0.3   0:08.40 openssl      

I checked the default CPUShares of the new services:

# systemctl show testSpeed | grep CPUShares
CPUShares=1024

At this point, I asked myself: Is RHEL 7 really different from RHEL 6 and all the Unix systems in general? Is RHEL 7 limiting any new service by default? After some time, I came to the conclusion that everything was exactly as before: there is no CPU limit (or any other limit) on your service by default.

Because I wanted to learn how CGroups were working, I decided to apply a CPU constraint:

# systemctl set-property testSpeed CPUShares=512
# systemctl daemon-reload

Note1: The daemon-reload option is only there to avoid a warning message but doesn’t change anything.
Note2: You don’t need to restart any service.

Now, the testSpeed service gets 33.3% of the CPU and the testSpeed2 gets 66.7%:

  PID USER      PR  NI S %CPU %MEM     TIME+ COMMAND      
24601 root      20   0 R 66.7  0.3   1:23.58 openssl      
24598 root      20   0 R 33.3  0.3   1:22.20 openssl      

What exactly happened?

By default, Systemd services are part of the system.slice hierarchy. When you set a CPU constraint on one service, you not only assign a CPU resource controller to this service but also to all the services in the system.slice hierarchy. It is not possible to apply a CPU constraint on one service without affecting all the others.

The CPUShares concept represents more a CPU priority than a limit.

Concerning the CPU usage, this is the result of a computation:

• testSpeed gets 33.3% because 512/(512+1024)=1/3
• testSpeed2 gets 66.7% because 1024/(512+1024)=2/3

It is better to know this behaviour because otherwise you would perhaps be very surprised at the worst moment!

I would like to thank Radoslaw Kujawa for helping me understand what I was suspecting.

Look at my CGroups page to get some other tips on this topic.

There are some activities on the CentOS 7 side:

• after a CentOS 7 (1503) alpha candidate for AArch64, there is now a CentOS 7 (1503) beta candidate for AArch64 platforms,
• a CentOS 7 (1503) beta candidate for i686 platforms has recently been announced,
• a CentOS Linux 7 Rolling ISO (1505) has been released.

More than providing a free alternative to RHEL 7, CentOS 7 is about to offer a wider choice of hardware platforms.

Many people don’t figure out why a Systemd service can be masked. They understand the need to enable it at boot and start it but why masking it?
Back to basics, a Systemd service can be:

• manually started with the systemctl start command,
• automatically started at boot with the systemctl enable command,
• dbus-activated by another Systemd service (NetworkManager can be triggered like this),
• socket-activated by some network activities (the printer service cups is a good example).

With this in mind, it becomes clear that masking a Systemd service with the systemctl mask command makes sense for a dbus-activated or a socket-activated service because in these cases stopping or disabling won’t definitively be enough!

Conclusion: If you want to be sure that a service will not start, mask it!

When a service doesn’t run as you would like, when you want to change its default configuration, Systemd offers you several options.

Firstly, you can copy the standard unit file from the /usr/lib/systemd/system directory into the /etc/systemd/system directory and then changes its content. The new unit file will take precedence over the default one. Don’t forget to run the systemctl daemon-reload command each time you create or modify a unit file

Also, if you want to make clear that you built the new unit file from the default one, you can add the following line at the beginning of the new unit file (here /etc/systemd/system/httpd.service):

.include /usr/lib/systemd/system/httpd.service
[Service]
CPUShares=500

Here you can read .include. It’s not a typo but the syntax required by Systemd.

The specified directive (here CPUShares) will override the previous configuration except if this directive allows multiple values, in this case, the new value will be added at the end. To avoid this situation, you will have to proceed in two steps: assign an empty value and then assign the desired value in another line.

Secondly, instead of copying the default unit file, you can create a directory of the same name with the .d suffix in the /etc/systemd/system directory and drop a file with the .conf suffix containing the settings you want to override. This is automatically done when using the systemctl set-property command. This method also allows to override settings coming from a unit file located in the /etc/systemd/system directory.

In our example, create the /etc/systemd/system/httpd.service.d directory, then the file called constraints.conf and paste the following lines into it:

[Service]
CPUShares=400

Thirdly, it is still possible to customize the environment before the service starts. You can specify a list of environment variables and their values with the Environment directive:

Environment="VAR1=word1 word2" VAR2=word3

You can also use a dedicated file with the EnvironmentFile directive. In the httpd case, you can see the following line in the /usr/lib/systemd/system/httpd.service file:

EnvironmentFile=/etc/sysconfig/httpd

Finally, don’t hesitate to run the systemd-delta command to know what was changed from the default configuration.

Who said that Systemd was not flexible!

To a non expert, Firewalld can sometimes appear a little bit strange and confusing.
Let’s take an example: you just set up an Apache webserver and want to configure a https virtual host.
Because it’s a test, you want to temporarily allow https on port tcp 443 to go through the firewall with the default zone:

# firewall-cmd --add-port=443/tcp
success
# firewall-cmd --list-ports
443/tcp

At this moment and because it was a temporary configuration, it would not have been a good idea to reload the firewall configuration, you would have lost the previous modification:

# firewall-cmd --reload
success
# firewall-cmd --list-ports
#

Later, you decide to make your configuration permanent:

# firewall-cmd --permanent --add-port=443/tcp
success

If you forget to reload the configuration, this is not going to work, at least until your next reboot!

# firewall-cmd --list-ports
#

But if you reload the firewall configuration, you get what you expected:

# firewall-cmd --reload
success
# firewall-cmd --list-ports
443/tcp

1st tip: A temporary configuration is directly activated without any need to reload the firewall configuration. But a permanent configuration requires a reload of the firewall configuration to work as expected. And it’s not the other way around!

This rule is verified for source management, port management, service management, masquerading and port forwarding but not for interface assignments to a zone or direct rules that are always directly activated whether temporary or not.

Concerning the firewall-cmd –list-ports command and lots of other commands used to check the configuration like firewall-cmd –list-services, firewall-cmd –query-masquerade, firewall-cmd –list-sources or firewall-cmd –list-all, you need to understand a subtile point:

• The firewall-cmd –list-ports command displays the current configuration, ie the list of the temporarily open ports and the permanently open ports already activated.
• The firewall-cmd –permanent –list-ports command only shows the permanent configuration, ie the list of the permanently open ports, activated or not!

2nd tip: During an exam, you need to use the –permanent option when applying most of the configurations and then reload the firewall configuration. However, when checking your configuration, you shouldn’t use the –permanent option if you want to get the correct information!

When working with services in RHEL 7, you can check if a service is currently running with the is-active option or enabled at boot time with the is-enabled option. But it is more difficult to know if the unit file is located in /usr/lib/systemd/system or /etc/systemd/system in the case of a custom service. It is also not obvious to know when the service was started or to get some CGroup information.
The systemctl status command brings all this information and even more.
Let’s take the httpd example and run the following command:

# systemctl status httpd
httpd.service - The Apache HTTP Server
Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled)
Active: active (running) since Tue 2015-04-21 11:48:23 CEST; 24h ago
Main PID: 2446 (httpd)
Status: "Total requests: 0; Current requests/sec: 0; Current traffic:   0 B/sec"
CGroup: /system.slice/httpd.service
└─2446 /usr/sbin/httpd -DFOREGROUND
└─2447 /usr/sbin/httpd -DFOREGROUND
└─2448 /usr/sbin/httpd -DFOREGROUND
└─2449 /usr/sbin/httpd -DFOREGROUND
└─2450 /usr/sbin/httpd -DFOREGROUND
└─2451 /usr/sbin/httpd -DFOREGROUND
└─2452 /usr/sbin/httpd -DFOREGROUND

Apr 21 11:48:23 server1.example.com systemd[1]: Starting The Apache HTTP Ser....
Apr 21 11:48:23 server1.example.com systemd[1]: Started The Apache HTTP Server.
Hint: Some lines were ellipsized, use -l to show in full.

The first line gives you a description of the service. You then learn that Systemd loaded the unit file from /usr/lib/systemd/system/httpd.service and the service is not enabled at boot time (disabled).
You next learn that the service has been running (active) for 24 hours and when was exactly the launch date.
The following line provides the Process ID.
In the case of the httpd service, you even get a quick activity report.
Then, the CGroup information is shown with the list of the child processes.
Finally, an extract from the journal is displayed with the most useful events.

Even though at first sight the systemctl status command seems to display too many lines, when you look at it, everything is there!

ISCSI configuration is a very tricky topic of the RHCE 7 exam.

If a RHCE candidate takes the exam without previous iScsi experience, he will fail!
Man pages are not enough to understand the intricacy of this subject and there is too much to read in a so limited time.

With iScsi configuration, you have to choose among several backstores, mainly fileio and block. Then, you have to set authentication, basic or CHAP (Challenge Handshake Authentication Protocol).

Also, you have to understand what is a target (the server side), an initiator (the client side), a LUN (Logical Unit Number), a portal (an entry point consisting of an ip address and a port), a TPG (Target Portal Group) and an ACL (Access Control List).

To get comfortable with this topic, you will have to practice iSCSI configuration at least 8 times: with each kind of backstores, with each kind of main authentication mechanisms and at least twice to get it memorized!

Good luck!

Today, Centos 7 (1503) aka CentOS 7.1 has just been released.
The distribution can now be downloaded from the CentOS website.

The Release Notes explain the major changes:

• As of March 2015 ABRT (>= 2.1.11-19.el7.centos.0.1) can report bugs directly to bugs.centos.org.
• Support for new processors (Intel Broadwell) and graphics (AMD Hawaii)
• Full support for LVM cache
• Ability to mount ceph block devices
• Updated Hyper-V network drivers
• New libguestfs features
• Full support for OpenJDK-1.8.0
• Improved clock stability (for PTP and NTP)
• Updated Networkmanager packages to version 1.0
• Updated docker to 1.4.1
• Updated OpenSSH to 6.6.1
• New package: Mozilla Thunderbird
• Update to numerous storage, network and graphics drivers
• Technology Preview: Support of the Btrfs file system, OverlayFS and the Cisco VIC kernel driver.

Reminder: This version is mandatory for RHCE candidates. It brings fixes for the Samba + Kerberos objective.

Happy download!