Loading...

VONDS ©

Lab 139: Remote Syslog & Logrotate Workflow

Back to Lab Index

Enable remote syslog reception over UDP/514 using a safe rsyslog drop-in, validate syntax, restart rsyslog, open the firewall, confirm the listener with ss, generate a test message with logger, verify it lands in /var/log/messages, then create and test a dedicated logrotate policy for /var/log/acmeapp.log.

troubleshooting logging core

Scenario

A team needs this host to receive syslog over UDP/514 from a legacy device. You must enable rsyslog UDP reception safely, allow it through the firewall, verify the listener, confirm logs arrive, then set up logrotate for a custom application log.

Safety note

Rsyslog config changes can break logging if you introduce syntax errors. Validate configuration with rsyslogd -N1 before restarting the service.

Objective

  • Create an rsyslog drop-in that loads imudp and listens on port 514.
  • Validate rsyslog config syntax with rsyslogd -N1.
  • Restart rsyslog to apply the listener configuration.
  • Allow 514/udp in firewalld permanently and reload.
  • Verify rsyslog is listening on UDP/514 with ss.
  • Generate a syslog test event with logger and confirm it lands in /var/log/messages.
  • Create a dedicated logrotate policy for /var/log/acmeapp.log.
  • Test the policy in debug mode with logrotate -d.

Concepts

  • Rsyslog drop-ins: isolated, reversible changes under /etc/rsyslog.d.
  • Safe rollout: validate config before restart using rsyslogd -N1.
  • Verifying listeners: confirm bound sockets and owning processes with ss.
  • Firewall workflow: make permanent changes, then apply with a reload.
  • Log rotation basics: per-app policies, retention, compression, and dry-run testing with -d.

Walkthrough

Step 1: Create the rsyslog UDP listener drop-in.
File 
sudo vi /etc/rsyslog.d/10-udp514.conf

Create a dedicated file under /etc/rsyslog.d so the change is isolated and easy to revert. Add the two lines below. 

module(load="imudp")
input(type="imudp" port="514")
Step 2: Validate rsyslog configuration syntax.
Command 
sudo rsyslogd -N1

This catches syntax issues before you restart the daemon. 

rsyslogd: version 8.2310.0, config validation run (level 1), master config /etc/rsyslog.conf
rsyslogd: End of config validation run. Bye.
Step 3: Restart rsyslog to apply the listener.
Command 
sudo systemctl restart rsyslog

Restart after validation so you do not take down logging with a broken config.

Step 4: Allow UDP/514 in firewalld permanently.
Command 
sudo firewall-cmd --permanent --add-port=514/udp
success
Step 5: Reload firewalld to apply permanent rules.
Command 
sudo firewall-cmd --reload
success
Ops habit

Permanent changes do nothing until you reload (or restart) firewalld. Always apply and then verify.

Step 6: Verify rsyslog is listening on UDP/514.
Command 
ss -lunp | grep ':514 '

You want to see an UNCONN UDP socket bound to 0.0.0.0:514 owned by rsyslog. 

UNCONN 0      0              0.0.0.0:514         0.0.0.0:*    users:(("rsyslogd",pid=1019,fd=7))
Step 7: Generate a test syslog message.
Command 
logger -p local0.notice 'test: UDP syslog receiving enabled'

This creates a clean, timestamped event you can immediately verify in the logs.

Step 8: Confirm the message appears in /var/log/messages.
Command 
sudo tail -n 8 /var/log/messages

Look for the listener acquisition line and your test message. 

Jan 25 07:14:21 lab139 rsyslogd[1019]: imudp: Acquired UDP socket, server will listen on port 514.
Jan 25 07:14:33 lab139 lab[pts/0]: test: UDP syslog receiving enabled
Step 9: Create a logrotate policy for /var/log/acmeapp.log.
File 
sudo vi /etc/logrotate.d/acmeapp

Use a dedicated policy file under /etc/logrotate.d so the application’s rotation rules are explicit and reviewable. 

/var/log/acmeapp.log {
    daily
    rotate 7
    compress
    missingok
    notifempty
    create 0640 root root
}
Step 10: Test the logrotate policy in debug mode.
Command 
sudo logrotate -d /etc/logrotate.d/acmeapp

Debug mode shows what logrotate would do without making changes. This is the safe way to validate a new policy. 

reading config file /etc/logrotate.d/acmeapp
Reading state from file: /var/lib/logrotate/logrotate.status
Handling 1 logs
rotating pattern: /var/log/acmeapp.log  after 1 days (7 rotations)
empty log files are not rotated, old logs are removed
consider log /var/log/acmeapp.log
log does not exist -- skipping
Signal to look for

“log does not exist -- skipping” is fine here. It proves the policy parsed correctly and would rotate the file once it exists.

Common breakpoints

rsyslog fails to restart

If systemctl restart rsyslog fails, re-run rsyslogd -N1, then check journalctl -u rsyslog for the exact parsing error and file/line location.

No UDP listener shown

If ss -lunp does not show :514, confirm your drop-in is readable and that your lines match exactly. Restart rsyslog after fixes, then re-check the socket.

Firewall open but logs do not arrive

Confirm the sender is targeting this host and that upstream network ACLs allow UDP/514. On the receiver, verify you are looking at the correct log file and confirm messages are being ingested with a fresh logger event.

logrotate policy parses but does nothing

A debug run will skip rotation if the file does not exist or is empty. Create the log file and add a few lines, then re-run logrotate -d to confirm the rotation logic.

Cleanup checklist

If this was a temporary test, remove the UDP drop-in and close the firewall port after you are done. In a real request, leave the configuration in place and document your verification output.

Commands 
sudo rsyslogd -N1
sudo systemctl status rsyslog --no-pager
ss -lunp | grep ':514 '
sudo firewall-cmd --list-ports
sudo tail -n 15 /var/log/messages
Success signal

ss shows rsyslog bound on UDP/514, your logger event appears in /var/log/messages, and logrotate parses your policy cleanly in debug mode.

Reference

  • /etc/rsyslog.d/10-udp514.conf: rsyslog drop-in for UDP reception.
    • module(load="imudp"): loads the UDP input module.
    • input(type="imudp" port="514"): binds rsyslog to UDP/514.
  • rsyslogd -N1: validate rsyslog configuration syntax safely.
    • -N1: runs a level-1 config validation (no daemon start).
  • systemctl restart rsyslog: apply config changes.
  • firewall-cmd --permanent --add-port=514/udp: open UDP/514 persistently.
    • --permanent: writes change to the active zone configuration.
    • --add-port=PORT/PROTO: adds an explicit port/protocol allowance.
  • firewall-cmd --reload: apply permanent firewall rules.
  • ss -lunp: verify UDP listener and owning process.
    • -l: show listening sockets.
    • -u: show UDP sockets.
    • -n: do not resolve names.
    • -p: show process info.
  • logger -p FACILITY.PRIORITY: generate a test syslog event.
    • -p: sets facility and priority (example: local0.notice).
  • tail -n N /var/log/messages: confirm the logging pipeline.
  • /etc/logrotate.d/acmeapp: per-app logrotate policy file.
  • logrotate -d FILE: debug a policy without changing files.
    • -d: debug (dry-run) mode.