Loading...

VONDS ©

Lab 137: Time Sync & Logging Workflow

Back to Lab Index

Validate time synchronization on a RHEL host using timedatectl, systemctl, and chronyc. Prove end-to-end logging by inspecting journald, validating rsyslog, and generating a test event with logger that lands in /var/log/messages.

troubleshooting time core

Scenario

A server is triggering time drift alerts, and responders need reliable logs for a fast incident review. You need to confirm time sync is actually working, apply a safe chrony configuration change with a backup-first workflow, and then prove the logging path by generating a new event and verifying it is written to disk.

Operator context

“Service is running” is not the same as “service is working.” Capture proof that the host has a selected time source, that unit logs exist in the journal, and that a fresh message traverses the syslog pipeline end-to-end.

Objective

  • Verify system time and NTP state with timedatectl.
  • Confirm chronyd is active and has at least one reachable source.
  • Back up /etc/chrony.conf, apply a makestep setting, and restart chrony.
  • Validate the service returns to an active state and selects a source after the change.
  • Confirm journal availability and review recent chronyd unit logs.
  • Verify rsyslog is active and writing to /var/log/messages.
  • Generate a test message with logger and confirm it is written to disk.

Concepts

  • High-level time state verification using timedatectl.
  • Service health checks and unit state inspection with systemctl.
  • Source selection and sync status validation using chronyc sources -v.
  • Backup-first configuration workflows and controlled restarts.
  • Journald as a source of truth for service timelines (journalctl).
  • Syslog pipeline proof using rsyslog, logger, and on-disk verification in /var/log/messages.

Walkthrough

Step 1: Check system time and NTP sync state.
Command 
timedatectl status

This is the fastest high-level signal: current time, time zone, and whether the host believes it is synchronized and running NTP. 

Local time: Sun 2026-01-25 07:12:19 EST
Universal time: Sun 2026-01-25 12:12:19 UTC
RTC time: Sun 2026-01-25 12:12:18
Time zone: America/New_York (EST, -0500)
System clock synchronized: yes
NTP service: active
RTC in local TZ: no
Step 2: Verify the chrony service is running.
Command 
sudo systemctl status chronyd --no-pager

Confirm the unit is active. If chrony is down, the clock can drift quickly and timestamps become unreliable. 

● chronyd.service - NTP client/server
Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; preset: enabled)
Active: active (running) since Sun 2026-01-25 06:58:41 EST; 13min ago
Main PID: 1042 (chronyd)
Step 3: Confirm chrony is syncing to a source.
Command 
chronyc sources -v

A selected source (^*) is the signal that the host is actively syncing rather than free-running. 

Number of sources = 2
MS Name/IP address         Stratum Poll Reach LastRx Last sample
===============================================================================
^* time1.example.net             2   6   377    32   -18us[ -41us] +/-  22ms
^+ time2.example.net             2   6   377    33   +12us[  -8us] +/-  24ms
Verify

If you do not see a ^* source, check reach (Reach column), DNS, and firewall rules, then review journalctl -u chronyd for failures.

Step 4: Back up the chrony configuration before making changes.
Command 
sudo cp -a /etc/chrony.conf /etc/chrony.conf.bak

This gives you an immediate rollback path if the change introduces an error or alters sync behavior unexpectedly.

Step 5: Apply a makestep setting and confirm it was written.
Command 
echo 'makestep 1.0 3' | sudo tee -a /etc/chrony.conf

makestep allows chrony to step the clock when the offset is large (commonly after boot or outages). Using tee makes the write explicit and visible. 

makestep 1.0 3
Step 6: Restart chronyd to apply the change.
Command 
sudo systemctl restart chronyd

A restart is a clean way to ensure the updated configuration is loaded.

Step 7: Confirm chronyd returned to an active state.
Command 
systemctl is-active chronyd
active

If this is not active, fix the configuration and confirm you can restart cleanly before continuing.

Step 8: Check journald disk usage.
Command 
journalctl --disk-usage

This is a quick health check for journal storage pressure and retention behavior. 

Archived and active journals take up 72.0M in the file system.
Step 9: Review recent chronyd logs from the journal.
Command 
journalctl -u chronyd -n 8 --no-pager

This confirms the restart is recorded and that chrony selected a source after the configuration change. 

Jan 25 06:58:41 lab137 systemd[1]: Starting NTP client/server...
Jan 25 06:58:41 lab137 chronyd[1042]: chronyd version 4.4 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +NTS)
Jan 25 06:58:43 lab137 chronyd[1042]: Selected source time1.example.net
Jan 25 07:11:58 lab137 systemd[1]: Stopping NTP client/server...
Jan 25 07:11:59 lab137 chronyd[1189]: Selected source time1.example.net
Step 10: Confirm rsyslog is running.
Command 
sudo systemctl status rsyslog --no-pager

If rsyslog is down, responders may think logs are missing on disk even though the journal still contains unit output. 

● rsyslog.service - System Logging Service
Loaded: loaded (/usr/lib/systemd/system/rsyslog.service; enabled; preset: enabled)
Active: active (running) since Sun 2026-01-25 06:58:38 EST; 13min ago
Main PID: 1019 (rsyslogd)
Step 11: Generate a test syslog message.
Command 
logger -p user.notice 'test: logging pipeline OK'

This creates a new event on demand so you can verify delivery rather than relying on historical entries.

Step 12: Confirm the message landed in /var/log/messages.
Command 
sudo tail -n 5 /var/log/messages

You are looking for the test line. This is the proof that rsyslog received the message and wrote it to disk. 

Jan 25 07:12:13 lab137 lab[pts/0]: test: logging pipeline OK

Common breakpoints

timedatectl shows unsynchronized

If System clock synchronized is no, verify chronyd is active, confirm sources are reachable with chronyc sources -v, and review unit logs with journalctl -u chronyd.

No selected chrony source

If you do not see a ^* source, focus on reach and connectivity. Check DNS resolution for the configured servers and confirm NTP traffic is permitted.

chronyd fails after config change

Roll back immediately using your backup, then restart. Confirm the unit is active before proceeding with logging validation.

logger message does not appear in /var/log/messages

Confirm rsyslog is active and check whether the host is configured to write facility/priority to /var/log/messages. Use journalctl -n to confirm the event exists in the journal even if it is not being forwarded.

Cleanup checklist

This lab is safe by default. If you edited /etc/chrony.conf, keep the backup and confirm chrony remains stable and synchronized after the restart.

Commands 
systemctl is-active chronyd
chronyc sources -v
journalctl -u chronyd -n 10 --no-pager
systemctl is-active rsyslog
Success signal

chronyd is active, a ^* source is selected, and new syslog events can be verified in the expected destination.

Reference

  • timedatectl status : View time zone, sync state, and NTP service state.
  • systemctl status <unit> --no-pager : Inspect service status and recent unit output.
    • chronyd: NTP client/server service.
    • rsyslog: Syslog daemon that writes logs to disk.
    • --no-pager: Prints output without paging.
  • systemctl is-active <unit> : Return a single-word unit state for scripting and fast checks.
  • chronyc sources -v : Show chrony sources and selection state.
    • ^*: Selected source (active sync target).
    • Reach: Reachability register; low reach indicates connectivity issues.
  • cp -a <src> <dst> : Copy files while preserving attributes.
    • -a: Archive mode (preserves permissions, timestamps, and context where applicable).
    • /etc/chrony.conf: Chrony configuration file.
  • tee -a <file> : Append stdin to a file and echo to stdout.
    • -a: Append instead of overwrite.
    • |: Pipes output from the left command into the right command.
  • journalctl --disk-usage : Report journald disk usage.
  • journalctl -u <unit> -n <N> --no-pager : View recent unit logs from the journal.
    • -u: Filter by unit.
    • -n <N>: Limit to the last N entries.
    • --no-pager: Prints output without paging.
  • logger -p <facility>.<priority> <message> : Generate a syslog event for pipeline testing.
    • -p: Set facility and priority (example: user.notice).
  • tail -n <N> <file> : View the last N lines of a file.
    • -n <N>: Number of lines to show.
    • /var/log/messages: Common rsyslog destination on RHEL-like systems.