RHEL7: Locate and interpret system log files and journals.

Share this link

Note: This is an RHCSA 7 exam objective.


Most of system log files are located in the /var/log directory due to SYSLOG default configuration (see /etc/rsyslog.conf file).

In addition, all SELinux events are written into the /var/log/audit/audit.log file.

With Systemd, new commands have been created to analyse logs at boot time and later.

Boot Process

Systemd primary task is to manage the boot process and provides informations about it.
To get the boot process duration, type:

# systemd-analyze
Startup finished in 422ms (kernel) + 2.722s (initrd) + 9.674s (userspace) = 12.820s

To get the time spent by each task during the boot process, type:

# systemd-analyze blame
7.029s network.service
2.241s plymouth-start.service
1.293s kdump.service
1.156s plymouth-quit-wait.service
1.048s firewalld.service
632ms postfix.service
621ms tuned.service
460ms iprupdate.service
446ms iprinit.service
344ms accounts-daemon.service
7ms systemd-update-utmp-runlevel.service
5ms systemd-random-seed.service
5ms sys-kernel-config.mount

Note: You will find additional information on this point in the Lennart Poettering’s blog.

Journal Analysis

In addition, Systemd handles the system event log, a syslog daemon is not mandatory any more.
The reasons behind Journald creation are explained in this Introduction to Journald.

To get the content of the Systemd journal, type:

# journalctl

To get all the events related to the crond process in the journal, type:

# journalctl /sbin/crond

Note: You can replace /sbin/crond by `which crond`.

Altenatively, to get all the events related to the crond process, you can also type:

# journalctl -u crond

To get all the events since the last boot, type:

# journalctl -b

To get all the events that appeared today in the journal, type:

# journalctl --since=today

To get all the events with a syslog priority of err, type:

# journalctl -p err

To get the 10 last events and wait for any new one (like tail -f /var/log/messages), type:

# journalctl -f

Permanent Storage

By default, Journald logs are stored in the /run/log/journal directory and disappear after a reboot.
To store Journald logs in a more permanent way, type:

# mkdir /var/log/journal
# echo "SystemMaxUse=50M" >> /etc/systemd/journald.conf
# systemctl restart systemd-journald 

Note: Setting the SystemMaxUse variable is necessary because otherwise 10% of the filesystem where the /var/log/journal directory is stored may be used at maximum by the journal.

Additional Resources

There is a tutorial dedicated to Systemd that presents additional information about the Systemd journal.
The rsyslog.com website provides some interesting information about rsyslogd (features, modules, plugins, etc).
Fedora Magazine also published an interesting article, Systemd: Using the journal.
Rainer Gerhards, rsyslog main author, discusses the arguments around the creation of Journald.
Nikolai Bezroukov wrote an authoritative page about Syslog.
The FreeIPA project (Identity Management with Kerberos, LDAP and Apache) provides several useful resources about Centralized Logging or Logging with Logstash/ElasticSearch/Kibana.
Logs can get flooded by some Systemd messages (starting sessions, etc). Red Hat provides a way to get rid of useless Systemd messages.

1 Star2 Stars3 Stars4 Stars5 Stars (1 votes, average: 5.00 out of 5)
11 comments on “RHEL7: Locate and interpret system log files and journals.
  1. fernandezcuesta says:

    To get all the events related to a unit I prefer to do (instead of the example above):

    # journalctl -u crond

    In general, -u for any other systemd unit.

  2. twostep says:

    “syslog daemon is not mandatory any more”, maybe during the exam, but during daily admin work, you still check /var/log/messages because not all logs are adjusted to the journal.

    • CertDepot says:

      I’m slightly surprised. I thought the systemd journal got more messages than the syslog daemon: it gets all boot and shutdown messages, when the syslog daemon doesn’t.

  3. twostep says:

    Option journalctl -b in the case when your logs are not permanently keep, in my opinion does not make sense. Please expand your article about a thread how to keep journal logs permanent, and then describe options journalctl -b -1,-2 etc. And as always thank you for a great job.

  4. manrizz says:

    Permanent storage for systemd journal as per Red Hat training, they recommend the following steps:
    # mkdir -p /var/log/journal
    # chown root:systemd-journal /var/log/journal
    # chmod 2755 /var/log/journal
    and the last step is to reboot the system or issue the following special signal: killall -USR1 systemd-journal

    can someone please explain what the main difference is and compare to the steps posted earlier:

    # mkdir /var/log/journal
    # echo “SystemMaxUse=50M” >> /etc/systemd/journald.conf
    # systemctl restart systemd-journald

    • Lisenet says:

      The first one sets the folder owner, group and setgid permissions while the second one does not, therefore in the latter case you end up with a folder that has the root group.

  5. RedHatter says:

    rsyslogd – In sander’s book the chapter on Configuring Logging has 70% of material on rsyslogd and only a couple pages on journald. He also says in the chapter “In the current state of RHEL 7, journald is not a replacement for rsyslog; it is just another way of logging information.” But since I don’t see much on rsyslogd here on this site, does it mean I don’t have to know it for the RHCSA/RHCE as long as I know journald? Trying to just focus on exams stuff for now.

  6. itninja says:

    Maybe it is worth of mentioning that we can find log paths of different types of messages in configuration file of /etc/rsyslog.conf – this daemon is modular and it accepts messages from journald daemon, but not detailed like in journal.

Leave a Reply

Upcoming Events (Local Time)

There are no events.

RHCSA7: Task of the day

Allowed time: 15 minutes.
Get an iso image of CentOS 7 (or a DVD) and set up a local repository.

RHCE7: Task of the day

Allowed time: 15 minutes.
Configure a httpd server with a password protected directory under the /var/www/html/private directory.

Follow me on Twitter

Poll for favorite RHEL 7 book

What is your favorite RHEL 7 book to prepare RHCSA & RHCE exams?

View Results

Loading ... Loading ...

Poll for most difficult RHCSA 7 topic

What do you think is the most difficult RHCSA 7 topic?

View Results

Loading ... Loading ...

Poll for most difficult RHCE 7 topic

What do you think is the most difficult RHCE 7 topic?

View Results

Loading ... Loading ...