Loading...

VONDS ©

Lab 127: Network Troubleshooting

Back to Lab Index

Restore outbound connectivity on a Linux VM that can reach its gateway but cannot reach the internet. Diagnose in order: addressing, local reachability, routing to public IPs, then DNS, and fix the missing default route and broken systemd-resolved configuration.

networking routing dns troubleshooting resolved

Scenario

You SSH into a Linux VM ( net-ops-127 ) after an app deploy. The node can reach the local gateway ( 192.168.56.1 ) but cannot reach the internet. Your job is to restore outbound connectivity and ensure DNS works again.

Operator context

Interface is eth0 and the gateway should be 192.168.56.1 . Validate public IP reachability before chasing DNS, then repair systemd-resolved using resolvectl with 1.1.1.1 and 8.8.8.8 .

Objective

  • Confirm an IPv4 address is present on eth0 .
  • Validate local reachability to the gateway.
  • Differentiate routing failures from DNS failures.
  • Detect a missing default route and add it correctly.
  • Use resolvectl to inspect and repair DNS server configuration.
  • Verify success with both IP and hostname tests.

Concepts

  • Layered troubleshooting: addressing, local path, routing, DNS.
  • Bypassing DNS by testing a public IP ( ping 1.1.1.1 ) before hostname tests.
  • Default route requirements for outbound connectivity ( default via ... ).
  • Runtime route repair with ip route add .
  • systemd-resolved inspection and link-scoped DNS using resolvectl .
  • NSS validation with getent hosts .

Walkthrough

Step 1 : Confirm IPv4 addressing on eth0 .
Command 
ip -4 addr show eth0

You want a valid inet address and the expected prefix (for example /24 ) before you proceed.

Step 2 : Prove the gateway is reachable.
Command 
ping -c 2 192.168.56.1

This confirms local L2/L3 connectivity. If this fails, fix the local network first (interface state, IP/subnet, L2 path, firewall).

Step 3 : Test routing with a public IP (bypass DNS).
Command 
ping -c 2 1.1.1.1

If this fails while the gateway ping succeeds, the problem is usually routing (often a missing default route) or an upstream policy/NAT issue.

Step 4 : Inspect the routing table.
Command 
ip route

You should see a line starting with default via ... . If it is missing, the host has no path to networks outside its local subnet.

Step 5 : Add the missing default route.
Command 
sudo ip route add default via 192.168.56.1 dev eth0
Operational note

This restores routing immediately, but may not persist across reboot depending on how networking is managed (NetworkManager, systemd-networkd, distro config files). The incident goal is restore first, then make it persistent through the correct manager.

Step 6 : Confirm the default route is present.
Command 
ip route

Confirm you now see default via 192.168.56.1 dev eth0 .

Step 7 : Re-test public IP reachability.
Command 
ping -c 2 1.1.1.1

Public IP tests should work before you move on to DNS.

Step 8 : Test DNS resolution with a hostname.
Command 
ping -c 1 example.com

If you see a name resolution error, DNS is misconfigured (or blocked) even though routing is now correct.

Step 9 : Inspect resolver configuration (systemd-resolved).
Command 
resolvectl status

Look at DNS servers assigned to eth0 . In this lab, a bad resolver is configured and needs to be replaced with known-good servers.

Step 10 : Set working DNS servers on eth0 .
Command 
sudo resolvectl dns eth0 1.1.1.1 8.8.8.8

This updates link-scoped DNS configuration for eth0 .

Step 11 : Validate name resolution using getent .
Command 
getent hosts example.com

getent confirms NSS resolution is working. This is a clean validation step before you rely on application behavior.

Step 12 : Final check with a hostname ping.
Command 
ping -c 1 example.com

If this works, outbound connectivity is restored and DNS is functioning.

Common breakpoints

Gateway ping fails

Check interface state, IP/prefix, and local path. Confirm the correct interface name with ip link and verify subnet settings.

Public IP ping fails even with a default route

Upstream NAT may be missing, an egress firewall may block ICMP, or the VM network may be isolated. In real systems, validate the path with traceroute and review policy.

DNS still fails after resolvectl dns

Confirm systemd-resolved is running, confirm the DNS servers appear under the correct link, then retest with resolvectl query example.com . Also consider firewall rules for UDP/TCP 53.

Route change disappears later

ip route add is runtime state. For persistence, set the default route in the network manager used by the system.

Cleanup checklist

This lab modifies runtime networking state. If you need to revert the changes, remove the default route and restore the prior link-scoped DNS configuration.

Commands 
sudo ip route del default via 192.168.56.1 dev eth0
sudo resolvectl revert eth0
ip route
resolvectl status
Success signal

The default route and DNS configuration reflect the intended state, and your reachability tests match expectations for the environment.

Reference

  • ip -4 addr show <if> : Confirms IPv4 address and prefix on an interface.
    • -4 : Limits output to IPv4 addresses.
  • ip route : Displays the routing table (including the default route).
  • ip route add default via <gw> dev <if> : Adds a default route for outbound traffic.
    • default : Route used when no more specific route matches.
    • via <gw> : Sets the next-hop gateway.
    • dev <if> : Sets the egress interface.
  • ping -c <n> <target> : Tests reachability to an IP or hostname.
    • -c <n> : Sends a fixed number of probes.
  • resolvectl status : Shows systemd-resolved status and per-link DNS servers.
  • resolvectl dns <if> <dns1> <dns2> : Sets DNS servers for a specific link.
    • dns : Updates link-scoped DNS server assignment.
  • resolvectl revert <if> : Reverts link-scoped resolver settings to the prior state.
  • getent hosts <name> : Validates NSS hostname resolution for a target name.