Loading...

VONDS ©

Lab 46: System Logs Monitor (/var/log)

Back to Lab Index

Investigate failed SSH attempts and missing scheduled-task output by reviewing common logs under /var/log . Use targeted filtering and simple counting to quantify activity and confirm what happened.

troubleshooting security services

Scenario

The web server team reports two issues: repeated SSH login failures and cron jobs that appear to run but do not produce expected output. Your task is to review logs in /var/log to confirm what happened, isolate relevant entries, and quantify activity.

Operator context

This is a fast triage workflow: verify the signal, collect evidence, and decide whether you are dealing with brute-force attempts, a misconfigured job, or a logging gap.

Objective

  • Confirm which log files record general system events and authentication activity.
  • Filter authentication logs to extract failed SSH attempts.
  • Confirm cron logging exists and review execution entries.
  • Monitor live log updates to observe events in real time.
  • Count SSH-related log entries to quantify activity for reporting.

Concepts

  • Traditional log layout under /var/log and distro differences in file naming.
  • Efficient large-file review using less and targeted filtering with grep .
  • Authentication log signal: “failed password” versus “accepted password” and invalid-user probes.
  • Cron execution evidence: proving a job triggered versus proving a job produced output.
  • Real-time observation using tail -f while reproducing an event.
  • Counting events with wc -l to create a first-pass metric.

Walkthrough

Step 1 : Identify the available logs under /var/log.
Command 
ls -lh /var/log

Start by confirming what files exist. Log file names and locations vary by distro and by whether the host is using rsyslog, syslog-ng, or journald forwarding. This step prevents you from searching for a file that is not used on the system. 

# Look for auth and system logs:
# auth.log, secure, syslog, messages, cron, cron.log
Step 2 : Review general system events.
Commands 
less /var/log/syslog
# OR (common on some distros)
less /var/log/messages

General logs provide context around service restarts, session activity, and system state changes. This is where you can correlate authentication spikes with reboots, network flaps, or service reloads. 

# Use / in less to search for keywords:
# sshd, CRON, systemd, error, warning
Step 3 : Extract failed SSH attempts from authentication logs.
Command 
grep "Failed password" /var/log/auth.log

Authentication logs are the source of truth for SSH login failures. Filtering on Failed password isolates failed attempts and helps you distinguish brute-force behavior from normal user error. 

# Common patterns:
# Failed password for invalid user  from  port  ssh2
# Failed password for  from  port  ssh2
Step 4 : Confirm cron logging exists and review execution entries.
Commands 
ls /var/log/cron*
# Then review:
less /var/log/cron

Cron activity is logged in different places depending on the distro and logging stack. Cron logs confirm that the scheduler executed a command under a specific user context. If output is missing, cron logs help you prove the trigger happened before you pivot to script paths, environment, and redirection. 

# Look for entries like:
# CRON[pid]: (user) CMD ()
Step 5 : Monitor log updates in real time while reproducing an event.
Commands 
tail -f /var/log/auth.log
# OR
tail -f /var/log/syslog

Live monitoring is useful when you are reproducing a login failure, validating that sshd is writing events, or proving that log updates are flowing. It gives immediate feedback without repeatedly reopening files. 

# Stop tail with Ctrl+C once you capture the evidence window.
Step 6 : Quantify SSH-related activity.
Command 
grep sshd /var/log/auth.log | wc -l

Counting entries provides a quick metric for reporting. This is not a perfect “attempt counter” (one login can generate multiple lines), but it is a useful first number when you are triaging under time pressure. 

# For a narrower count of failures only:
# grep "Failed password" /var/log/auth.log | wc -l

Common breakpoints

auth.log does not exist on this host

Some distros log authentication events to /var/log/secure instead. Confirm the available files under /var/log and re-run the same filters against the correct log.

Cron log is missing or empty

Cron may be logging into the general syslog stream or only into journald. Check /var/log/syslog or /var/log/messages for CRON entries before assuming the scheduler is not running.

grep output is too noisy or too sparse

Tune the pattern. Target "Failed password" for failures or filter by a known username or IP address. If you suspect rotation, check rotated logs (for example auth.log.1 ) before concluding activity stopped.

tail -f shows no updates during a reproduced event

Confirm you are following the correct file and that the logging pipeline is functioning. If the host uses journald as the primary sink, text logs may not update the way you expect.

Cleanup checklist

This lab is read-only. Your cleanup is to capture evidence (filtered outputs and timestamps) and stop any live-follow sessions once you have the required window.

Commands 
ls -lh /var/log
grep "Failed password" /var/log/auth.log
ls /var/log/cron*
less /var/log/cron
Success signal

You can point to the exact log lines that prove the authentication failures and cron execution activity, and you have a count suitable for reporting.

Reference

  • ls -lh /var/log : Lists log files with sizes and timestamps.
    • -l : Long listing format.
    • -h : Human-readable sizes.
    • /var/log : Standard directory for traditional text logs.
  • less <file> : Reads a log file interactively.
    • / : Search forward for a pattern.
    • q : Quit.
  • grep <pattern> <file> : Filters lines matching a pattern.
    • "Failed password" : Common sshd failure string in authentication logs.
  • ls /var/log/cron* : Confirms cron log files and rotated variants exist.
    • * : Shell glob matching rotated files (for example cron.1 ).
  • tail -f <file> : Follows a file as new lines are appended.
    • -f : Follow updates as the file grows.
  • grep <pattern> <file> | wc -l : Counts matching log lines.
    • | : Pipes output from the left command into the right command.
    • wc -l : Counts lines.