Loading...

VONDS ©

Lab 110: Monitor Hardware Events

Back to Lab Index

Monitor real hardware changes by combining kernel log inspection with live udev event tracking. Use readable timestamps, targeted filtering, udev property queries, and controlled udev triggers to validate device identity and rule execution.

troubleshooting hardware udev

Scenario

A user reports intermittent issues with a USB device and occasional storage detection delays at boot. You need to gather evidence of device connect events, confirm how the kernel and udev classify the device, and validate that udev rules can be triggered and settled predictably.

Operator context

For hardware issues, timestamps and device identity matter. Capture reliable evidence, reduce noise through filtering, and confirm device attributes directly from udev and sysfs before changing system state.

Objective

  • Inspect recent kernel messages and isolate relevant events.
  • Use human-readable timestamps for incident-quality log review.
  • Filter kernel logs for USB-related activity.
  • Monitor live kernel and udev events with udevadm monitor .
  • Query udev properties for a device node with udevadm info .
  • Trigger udev processing for the block subsystem and wait for completion.
  • Confirm device identity by reading vendor and product IDs from sysfs.

Concepts

  • Kernel ring buffer vs udev processing: what each stream proves.
  • Why readable timestamps matter for tickets and incident notes.
  • How udev properties map a device node to a sysfs path used by rules.
  • Why udevadm settle makes trigger workflows repeatable.

Walkthrough

Step 1 : Show the last 20 kernel messages.
Command 
dmesg | tail -n 20

This is the quickest way to see what the kernel has logged most recently. It can confirm whether a device was detected, whether a driver bound successfully, and whether errors are present. 

# Expected patterns:
# usb ... new ... device
# ata... link up
# driver init lines
Step 2 : Use human-readable timestamps and scope output.
Command 
dmesg -T | tail -n 5

Human-readable timestamps help with incident notes and correlate events across logs. Use this format when you plan to paste evidence into a ticket or write-up. 

# Example:
# [Tue Aug 19 12:08:03 2025] ...
Step 3 : Filter for USB-related messages.
Command 
dmesg -T | grep -i usb | tail -n 10

Filtering reduces noise so you can focus on device detection and enumeration events. This is often where you find vendor and product IDs, device speed, and interface initialization. 

# Expected patterns:
# usb ... new ... device
# idVendor=.... idProduct=....
Step 4 : Monitor live kernel and udev events.
Command 
udevadm monitor --kernel --udev

This is your live event stream. Kernel uevents show what the kernel emits, and udev events show what user space receives and processes. On device connect, you should see matching add events in both streams. 

# Expected patterns:
# KERNEL[...] add ... (usb)
# UDEV  [...] add ... (usb)
Step 5 : Query udev properties for a device node.
Command 
udevadm info --query=all --name=/dev/sda

udevadm info prints resolved udev properties for a device node. This is where you confirm bus type, model, serial, and the canonical sysfs path used by rules. 

# Key fields to look for:
# E: DEVNAME=/dev/sda
# E: ID_BUS=...
# E: ID_MODEL=...
# E: ID_SERIAL=...
# E: DEVPATH=/devices/...
Step 6 : Trigger udev for block devices and wait for completion.
Command 
udevadm trigger --subsystem-match=block; udevadm settle

udevadm trigger replays events so rules can re-evaluate existing devices. udevadm settle waits until udev finishes processing, which makes scripted verification steps repeatable.

Step 7 : Confirm USB vendor and product IDs from sysfs.
Command 
cat /sys/bus/usb/devices/1-1/idVendor

sysfs is the kernel’s live device tree. Vendor and product IDs from sysfs are strong evidence of device identity without relying on log parsing.

Command 
cat /sys/bus/usb/devices/1-1/idProduct
# Example:
046d
c534

Common breakpoints

dmesg output looks empty or truncated

The ring buffer may have rotated. Capture evidence sooner, or use your system log pipeline for longer retention.

udevadm monitor shows kernel events but not udev events

udev may not be running or events may be filtered by your environment. Confirm your init system and udev status.

/sys/bus/usb/devices/1-1 does not exist

USB device paths vary by host. Identify the correct sysfs path from udevadm info using E: DEVPATH=... and read IDs from that device directory.

Cleanup checklist

This lab is read-only unless you choose to run triggers. If you started a monitor session, stop it with Ctrl+C .

Reference

  • dmesg : Reads the kernel ring buffer (recent kernel messages).
    • -T or --ctime : Human-readable timestamps.
  • udevadm monitor : Watches live device events.
    • --kernel: Show kernel uevents.
    • --udev: Show udev-processed events.
  • udevadm info --query=all --name=/dev/<node> : Displays resolved udev properties and sysfs path for a device node.
  • udevadm trigger : Retriggers events so udev rules can re-run against existing devices.
    • --subsystem-match=<name> : Restrict triggers to a subsystem (example: block ).
  • udevadm settle : Waits until udev has finished processing queued events.
  • /sys : sysfs virtual filesystem exposing live kernel object state (device IDs, attributes, drivers).