Loading...

VONDS ©

Lab 9: Diagnose and Manage a Faulty Service (Systemd)

Lab Index

Diagnose why a systemd service fails using status and journal evidence. Apply the fix, restart cleanly, confirm runtime health, and ensure the service is enabled for future boots.

services troubleshooting core

Scenario

The nginx.service is failing to start on boot. You must investigate the cause, fix it, and ensure it runs now and on future boots.

Operator context

This is a standard production workflow: confirm failure, pull the exact error from the journal, resolve the root cause, then prove it stays healthy after restart and across reboots.

Objective

  • Confirm the service failure using systemctl status .
  • Collect detailed failure evidence using journalctl -xeu .
  • Identify the cause (port conflict) from logs.
  • Restart the service and confirm it transitions to active (running) .
  • Ensure the service starts on boot using systemctl enable .

What You’ll Practice

  • Investigating services using systemctl status .
  • Reading unit-specific logs using journalctl -xeu <unit> .
  • Recognizing a port binding conflict ( Address already in use ) as a failure mode.
  • Restarting services using systemctl restart and validating runtime state.
  • Enabling services at boot using systemctl enable and recognizing the created target symlink.

Walkthrough

Step 1 : Check the service status.
Command 
systemctl status nginx

This is your initial triage view: loaded state, enablement, active state, and the failing ExecStart context. If it is failing, your next step is the journal for the exact error text. 

● nginx.service - A high performance web server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled)
   Active: failed (Result: exit-code) since ...
   Docs: man:nginx(8)
   Process: 1356 ExecStart=/usr/sbin/nginx -g 'daemon on;' (code=exited, status=1/FAILURE)
Step 2 : Pull detailed logs for the unit.
Command 
journalctl -xeu nginx

This filters the journal to just the nginx unit while showing the most relevant recent errors. The goal is to extract the root-cause line, not to guess. 

Jul 18 12:43:01 nginx[1356]: nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
Jul 18 12:43:01 systemd[1]: nginx.service: Failed with result 'exit-code'.
Diagnosis

The bind failure means something else is already listening on port 80. After resolving the conflict, restart nginx.

Step 3 : Restart nginx after the conflict is resolved.
Command 
sudo systemctl restart nginx

A restart re-runs the unit’s start sequence. If the root cause is actually fixed, the unit should transition to active (running) .

Step 4 : Verify nginx is running.
Command 
systemctl status nginx
● nginx.service - A high performance web server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled)
   Active: active (running) since ...
Step 5 : Ensure nginx starts on boot.
Command 
sudo systemctl enable nginx

Enabling a unit creates the appropriate symlink in the target’s wants directory (commonly multi-user.target.wants for server services). This is how you make behavior persist across reboots. 

Created symlink /etc/systemd/system/multi-user.target.wants/nginx.service → /lib/systemd/system/nginx.service

Reference

  • systemctl status <unit> : Shows loaded state, active state, recent logs, and failure reason for a systemd unit.
  • journalctl -xeu <unit> : Displays recent, detailed journal entries for a specific systemd unit.
    • -x : Adds explanatory text when available.
    • -e : Jumps to the end of the log.
    • -u : Filters by unit.
  • systemctl restart <unit> : Restarts the unit immediately.
  • systemctl enable <unit> : Enables the unit to start automatically on boot (creates a symlink under the appropriate target wants directory).