Skip to content

Systemctl Troubleshooting

Jump to bottom
Mattscreative edited this page Dec 5, 2025 · 3 revisions

Systemctl Troubleshooting for Beginners

Table of Contents

  1. 📝 What is systemctl?
  2. ⚡ Basic Commands
  3. 🔍 Checking Service Status
  4. :desktop: Listing Services
  5. 🔧 Controlling Services
  1. 🔑 Enabling and Disabling Services
  1. 📁 Viewing Service Information
  1. ⏰ Working with Timers
  1. ⚠️ Troubleshooting Failed Services
  1. 🔎 Advanced Troubleshooting
  1. 💡 Common Troubleshooting Scenarios
  1. 💾 Service File Locations
  2. ⌨️ Quick Reference
  1. Summary

📝 What is systemctl?

  • systemctl is the command-line tool for managing systemd services and units
  • It allows you to control, monitor, and troubleshoot system services
  • systemd is the system and service manager used by most modern Linux distributions
  • Services are called "units" in systemd terminology

What systemctl can do:

  • Start, stop, restart, and reload services
  • Check service status and health
  • Enable/disable services to start at boot
  • View service logs
  • Check service dependencies
  • Troubleshoot failed services

⚡ Basic Commands

Getting Help

systemctl --help

Shows all available systemctl commands and options.

systemctl status

Shows the overall system status, including:

  • System state (running, degraded, etc.)
  • Number of loaded units
  • Number of failed units
  • System uptime
  • systemd version

Example output:

● fedora
    State: running
    Units: 746 loaded (incl. loaded aliases)
     Jobs: 0 queued
   Failed: 0 units
    Since: Thu 2025-11-13 03:33:58 AST; 11h ago
  systemd: 258.1-1.fc43

🔍 Checking Service Status

Viewing a Specific Service Status

systemctl status servicename

What this does:

  • Shows detailed information about a service
  • Displays current state (active/inactive, running/stopped)
  • Shows recent log entries
  • Indicates if the service is enabled to start at boot

Common service names:

  • sshd - SSH server
  • nginx - Nginx web server
  • httpd or apache2 - Apache web server
  • libvirtd - Virtualization daemon
  • NetworkManager - Network management
  • docker - Docker daemon
  • postgresql - PostgreSQL database

Example:

systemctl status sshd

What you'll see:

  • Service state (active/inactive, running/stopped)
  • Whether it's enabled at boot
  • Recent log entries
  • Process information
  • Loaded configuration file path

Quick Status Checks

Check if a service is active (running):

systemctl is-active servicename

Output:

  • active - Service is running
  • inactive - Service is not running
  • Exit code 0 = active, non-zero = inactive

Check if a service is enabled (starts at boot):

systemctl is-enabled servicename

Output:

  • enabled - Service will start at boot
  • disabled - Service will not start at boot
  • masked - Service is prevented from starting (even manually)
  • Exit code 0 = enabled, non-zero = disabled/masked

Check if a service has failed:

systemctl is-failed servicename

Output:

  • failed - Service has failed
  • Empty output - Service is not in failed state
  • Exit code 0 = not failed, non-zero = failed

:desktop: Listing Services

List All Running Services

systemctl list-units --type=service --state=running

What this does:

  • Shows all services that are currently running
  • Displays service name, load state, active state, and description

Example output:

  UNIT                       LOAD   ACTIVE SUB     DESCRIPTION
  abrt-journal-core.service  loaded active running ABRT coredumpctl message creator
  abrt-oops.service          loaded active running ABRT kernel log watcher
  abrt-xorg.service          loaded active running ABRT Xorg log watcher
  abrtd.service              loaded active running ABRT Daemon

What each column means:

  • UNIT: Service name
  • LOAD: Whether the unit file is loaded
  • ACTIVE: High-level activation state
  • SUB: Low-level activation state (running, dead, etc.)
  • DESCRIPTION: What the service does

List All Failed Services

systemctl list-units --type=service --state=failed

What this does:

  • Shows only services that have failed
  • Critical for troubleshooting system issues
  • Helps identify what's broken

When to use it:

  • System is behaving strangely
  • Something isn't working
  • After system updates
  • Troubleshooting boot issues

If no failed services:

  UNIT LOAD ACTIVE SUB DESCRIPTION

0 loaded units listed.

List All Services (All States)

systemctl list-units --type=service --all

What this does:

  • Shows all services regardless of state
  • Includes inactive, failed, and running services
  • Useful for finding a specific service

List Enabled Services (Start at Boot)

systemctl list-unit-files --type=service --state=enabled

What this does:

  • Shows all services that are enabled to start at boot
  • Displays the unit file state and preset state

Example output:

UNIT FILE                             STATE   PRESET
abrt-journal-core.service             enabled enabled
abrt-oops.service                     enabled enabled
abrt-vmcore.service                   enabled enabled
abrt-xorg.service                     enabled enabled

🔧 Controlling Services

Starting a Service

sudo systemctl start servicename

What this does:

  • Starts a service immediately
  • Does not enable it to start at boot (use enable for that)
  • Requires root privileges (sudo)

Example:

sudo systemctl start sshd

When to use it:

  • Service is stopped and you need it running
  • Testing if a service works
  • Temporarily starting a disabled service

Stopping a Service

sudo systemctl stop servicename

What this does:

  • Stops a running service immediately
  • Does not disable it from starting at boot
  • Requires root privileges

Example:

sudo systemctl stop sshd

When to use it:

  • Service is misbehaving and needs to be stopped
  • Temporarily disabling a service
  • Before making configuration changes

Restarting a Service

sudo systemctl restart servicename

What this does:

  • Stops and then starts a service
  • Applies configuration changes
  • Useful after editing service configuration files

Example:

sudo systemctl restart nginx

When to use it:

  • After changing service configuration
  • Service is acting strange and needs a fresh start
  • Applying updates

Reloading a Service

sudo systemctl reload servicename

What this does:

  • Reloads service configuration without stopping it
  • Service continues running during reload
  • Not all services support reload (some require restart)

Example:

sudo systemctl reload nginx

When to use it:

  • After configuration changes
  • When you want to avoid service downtime
  • Services that support reload (check with systemctl status)

Try-Restart (Restart Only If Running)

sudo systemctl try-restart servicename

What this does:

  • Restarts the service only if it's currently running
  • If the service is stopped, it does nothing
  • Safer than regular restart

When to use it:

  • Restarting a service that may or may not be running
  • Scripts where you're not sure of the service state

Reload-or-Restart

sudo systemctl reload-or-restart servicename

What this does:

  • Tries to reload first (if supported)
  • Falls back to restart if reload is not supported
  • Best of both worlds

When to use it:

  • Applying configuration changes
  • When you're not sure if the service supports reload

🔑 Enabling and Disabling Services

Enable Service to Start at Boot

sudo systemctl enable servicename

What this does:

  • Configures the service to start automatically at boot
  • Does not start the service immediately (use start for that)
  • Creates symlinks to enable the service

Example:

sudo systemctl enable sshd

When to use it:

  • You want a service to start automatically on boot
  • Setting up a new service
  • After installing a service that should run at boot

Disable Service from Starting at Boot

sudo systemctl disable servicename

What this does:

  • Prevents the service from starting at boot
  • Does not stop the service if it's currently running
  • Removes symlinks that enable the service

Example:

sudo systemctl disable sshd

When to use it:

  • Service should not start automatically
  • Reducing boot time
  • Disabling unnecessary services

Enable and Start Together

sudo systemctl enable --now servicename

What this does:

  • Enables the service to start at boot
  • Starts the service immediately
  • Combines enable and start in one command

Example:

sudo systemctl enable --now sshd

When to use it:

  • Setting up a new service that should run now and at boot
  • Quick way to enable and start in one step

Disable and Stop Together

sudo systemctl disable --now servicename

What this does:

  • Disables the service from starting at boot
  • Stops the service immediately
  • Combines disable and stop in one command

When to use it:

  • Completely disabling a service
  • Removing a service from the system

Masking a Service

sudo systemctl mask servicename

What this does:

  • Prevents a service from being started (even manually)
  • Creates a symlink to /dev/null
  • Stronger than disable - service cannot be started at all

When to use it:

  • Completely preventing a service from running
  • When disable isn't enough
  • Troubleshooting service conflicts

Unmasking:

sudo systemctl unmask servicename

📁 Viewing Service Information

View Service Properties

systemctl show servicename

What this does:

  • Shows all properties of a service
  • Displays detailed configuration information
  • Useful for troubleshooting

Example:

systemctl show sshd

What you'll see:

  • Service ID and names
  • Dependencies (Requires, Wants, After, Before)
  • Documentation links
  • Description
  • Load state
  • Active state
  • And much more

View Service Dependencies

systemctl list-dependencies servicename

What this does:

  • Shows what services this service depends on
  • Displays the dependency tree
  • Helps understand service relationships

Example:

systemctl list-dependencies sshd

What you'll see:

  • Services that must be running before this service
  • Services that this service wants (optional)
  • Target units (like network.target, sysinit.target)

Reverse dependencies (what depends on this service):

systemctl list-dependencies --reverse servicename

View Service Configuration File

systemctl cat servicename

What this does:

  • Shows the service unit file content
  • Displays the actual configuration
  • Includes drop-in files (overrides)

When to use it:

  • Understanding service configuration
  • Checking what a service does
  • Before making changes

View Service Logs

systemctl status servicename

Shows recent log entries in the status output.

For more detailed logs:

journalctl -u servicename

For real-time log following:

journalctl -u servicename -f

See the Journalctl Troubleshooting Guide for more log viewing options.

⏰ Working with Timers

List All Timers

systemctl list-timers

What this does:

  • Shows all systemd timer units
  • Displays when timers will run next
  • Shows last run time

Example output:

NEXT                          LEFT LAST                              PASSED UNIT                         ACTIVATES
Thu 2025-11-13 16:04:14 AST  44min Thu 2025-11-13 12:08:03 AST 3h 11min ago dnf-makecache.timer          dnf-makecache.service
Fri 2025-11-14 00:00:00 AST     8h Thu 2025-11-13 07:34:00 AST       7h ago unbound-anchor.timer         unbound-anchor.service

What each column means:

  • NEXT: When the timer will run next
  • LEFT: Time until next run
  • LAST: When it last ran
  • PASSED: Time since last run
  • UNIT: Timer unit name
  • ACTIVATES: Service that the timer activates

When to use it:

  • Checking scheduled tasks
  • Understanding system maintenance schedules
  • Troubleshooting automated tasks

⚠️ Troubleshooting Failed Services

Check for Failed Services

systemctl --failed

What this does:

  • Lists all failed services
  • Quick way to see what's broken
  • Equivalent to list-units --state=failed

Reset Failed State

sudo systemctl reset-failed servicename

What this does:

  • Clears the "failed" state from a service
  • Useful after fixing a service issue
  • Resets the service status

When to use it:

  • After fixing a service configuration
  • After resolving dependency issues
  • Cleaning up service state

Reset all failed services:

sudo systemctl reset-failed

Reload systemd Configuration

sudo systemctl daemon-reload

What this does:

  • Reloads systemd manager configuration
  • Rereads all unit files
  • Required after creating or modifying unit files

When to use it:

  • After creating a new service file
  • After modifying a service unit file
  • After installing a new service
  • Before starting a newly created service

Important: Always run this after editing service files in /etc/systemd/system/

🔎 Advanced Troubleshooting

View System State

systemctl status

Shows overall system health.

Check if system is degraded:

systemctl is-system-running

Possible outputs:

  • running - System is running normally
  • degraded - Some units failed but system is operational
  • maintenance - System is in maintenance mode
  • initializing - System is still booting
  • starting - System is starting up
  • stopping - System is shutting down

View Service Dependencies Graph

systemctl list-dependencies servicename --all

What this does:

  • Shows complete dependency tree
  • Includes all dependencies recursively
  • Shows both required and wanted dependencies

Check Service Logs with journalctl

journalctl -u servicename

Filter by priority:

journalctl -u servicename -p err

Follow logs in real-time:

journalctl -u servicename -f

Since a specific time:

journalctl -u servicename --since "1 hour ago"

See the Journalctl Troubleshooting Guide for comprehensive log viewing options.

💡 Common Troubleshooting Scenarios

Scenario 1: Service Won't Start

Problem: A service fails to start.

Solution:

  1. Check service status:

    systemctl status servicename

  2. Check for errors in logs:

    journalctl -u servicename -p err

  3. Check service dependencies:

    systemctl list-dependencies servicename

  4. Verify service file:

    systemctl cat servicename

  5. Try starting with verbose output:

    systemctl start servicename
systemctl status servicename

Scenario 2: Service Keeps Crashing

Problem: Service starts but immediately stops.

Solution:

  1. Check service status for error messages:

    systemctl status servicename

  2. Follow logs in real-time:

    journalctl -u servicename -f

  3. Check for configuration errors:

    systemctl cat servicename

  4. Check system resources:

    free -h
df -h

Scenario 3: Service Not Starting at Boot

Problem: Service works when started manually but doesn't start at boot.

Solution:

  1. Check if service is enabled:

    systemctl is-enabled servicename

  2. If disabled, enable it:

    sudo systemctl enable servicename

  3. Check service dependencies:

    systemctl list-dependencies servicename

  4. Verify service file for After/Before directives:

    systemctl cat servicename

Scenario 4: Finding What's Using a Port

Problem: Port is already in use, preventing service from starting.

Solution:

  1. Find what's using the port (example for port 80):

    sudo ss -tlnp | grep :80

  2. Check the service status:

    systemctl status servicename

  3. Stop the conflicting service if needed:

    sudo systemctl stop conflicting-service

Scenario 5: Service Takes Too Long to Start

Problem: Service is slow to start.

Solution:

  1. Check service status and startup time:

    systemctl status servicename

  2. Check dependencies:

    systemctl list-dependencies servicename

  3. Check system load:

    uptime
systemctl status

  4. Review logs for slow operations:

    journalctl -u servicename --since "boot"

Scenario 6: Understanding Service Dependencies

Problem: Need to understand why a service requires another service.

Solution:

  1. View dependencies:

    systemctl list-dependencies servicename

  2. View reverse dependencies:

    systemctl list-dependencies --reverse servicename

  3. View service properties:

    systemctl show servicename | grep -i depend

💾 Service File Locations

System service files:

  • /etc/systemd/system/ - Local system service files (highest priority)
  • /run/systemd/system/ - Runtime service files
  • /usr/lib/systemd/system/ - Package-installed service files (lowest priority)

User service files:

  • ~/.config/systemd/user/ - User service files

Drop-in files (overrides):

  • /etc/systemd/system/servicename.service.d/ - Service-specific overrides
  • /etc/systemd/system/servicename.service.d/override.conf - Common override file

After editing service files: Always run:

sudo systemctl daemon-reload

⌨️ Quick Reference

Status Commands

systemctl status                    # Overall system status
systemctl status servicename        # Service status
systemctl is-active servicename     # Quick active check
systemctl is-enabled servicename    # Quick enabled check
systemctl is-failed servicename     # Quick failed check
systemctl --failed                  # List failed services

Control Commands

sudo systemctl start servicename    # Start service
sudo systemctl stop servicename     # Stop service
sudo systemctl restart servicename  # Restart service
sudo systemctl reload servicename   # Reload service
sudo systemctl reload-or-restart servicename  # Reload or restart

Enable/Disable Commands

sudo systemctl enable servicename           # Enable at boot
sudo systemctl disable servicename          # Disable at boot
sudo systemctl enable --now servicename     # Enable and start
sudo systemctl disable --now servicename    # Disable and stop
sudo systemctl mask servicename             # Mask service
sudo systemctl unmask servicename           # Unmask service

Listing Commands

systemctl list-units --type=service                    # All services
systemctl list-units --type=service --state=running    # Running services
systemctl list-units --type=service --state=failed     # Failed services
systemctl list-unit-files --type=service --state=enabled  # Enabled services
systemctl list-timers                                  # All timers

Information Commands

systemctl show servicename                    # Service properties
systemctl list-dependencies servicename      # Service dependencies
systemctl cat servicename                    # Service file content
journalctl -u servicename                    # Service logs

Maintenance Commands

sudo systemctl daemon-reload                  # Reload systemd config
sudo systemctl reset-failed servicename      # Reset failed state
sudo systemctl reset-failed                   # Reset all failed

Summary

This guide covered:

  1. Basic Commands:
  • Getting help with --help
  • Checking overall system status
  • Viewing service status
  1. Status Checking:
  • status - Detailed service status
  • is-active - Quick active check
  • is-enabled - Quick enabled check
  • is-failed - Quick failed check
  1. Listing Services:
  • Running services
  • Failed services
  • Enabled services
  • All services
  1. Controlling Services:
  • Start, stop, restart, reload
  • Try-restart and reload-or-restart
  • Enable and disable at boot
  • Mask and unmask services
  1. Service Information:
  • Viewing service properties
  • Checking dependencies
  • Viewing configuration files
  • Viewing service logs
  1. Timers:
  • Listing scheduled tasks
  • Understanding timer output
  1. Troubleshooting:
  • Finding failed services
  • Resetting failed state
  • Reloading systemd configuration
  • Common troubleshooting scenarios
  1. Service File Locations:
  • Where service files are stored
  • Drop-in files and overrides

Next Steps:

  • Practice with services on your system
  • Combine with journalctl for comprehensive troubleshooting
  • Learn about creating custom service files
  • Explore timer units for scheduling tasks

For more advanced topics, see the Journalctl Troubleshooting Guide for detailed log analysis.

Clone this wiki locally