Service

Service Analysis in Incident Response (Linux Systems)

Services in Linux play a vital role in the system's operation. Attackers often exploit this functionality to maintain persistence, allowing them to regain access after system reboots. Here's a comprehensive guide to detecting and analyzing malicious services during incident response.


Key Concepts of Linux Services

  1. Service Status Types:

    • Active (running): Service is currently running.

    • Active (exited): Service has completed its task and exited.

    • Active (waiting): Service is running, waiting for a trigger.

    • Inactive: Service is not running.

    • Enabled: Configured to start at boot.

    • Disabled: Will not start automatically at boot.


1. Listing and Investigating Services

Using service Command

service --status-all
  • Lists all available services along with their status.

Using systemctl Command

systemctl list-units --type=service
  • Lists currently loaded services with their statuses.

List All Enabled Services (Startup Services)

systemctl list-unit-files --type=service | grep enabled

2. Identifying Suspicious or Unknown Services

Attackers often create services with inconspicuous names or names similar to legitimate services (e.g., cron_service vs. cron).

Search for Specific Services

If you suspect a service:

sudo systemctl status SERVICE_NAME

This shows:

  • Service status.

  • Executable path.

  • Recent logs.

Inspect the Service’s Unit File

cat /lib/systemd/system/SERVICE_NAME.service

Look for:

  • ExecStart: Path to the executable/script that starts the service.

  • Restart: Conditions under which the service restarts.

  • Environment: Variables passed to the service.

Analyze the Unit File Modification Details

stat /lib/systemd/system/SERVICE_NAME.service
  • Modified Time: Can indicate unauthorized changes.

  • Access Time: Shows when it was last accessed.


3. Review Historical and Real-Time Logs

Logs can reveal service creation, modifications, and executions.

Analyze System Logs

To check for service-related logs:

sudo journalctl | grep service

Review Logs for a Specific Service

sudo journalctl -u SERVICE_NAME

Use time filters for more targeted analysis:

sudo journalctl -u SERVICE_NAME --since "2023-11-10" --until "2023-11-13"

Check Logs for All Activities in a Time Range

sudo journalctl --since "2023-11-10 00:00" --until "2023-11-13 23:59"

4. Investigate and Isolate Suspicious Services

Check Service Dependencies

Some malicious services rely on other services or scripts:

systemctl list-dependencies SERVICE_NAME

Inspect Processes Started by the Service

ps aux | grep <PID>
  • Find and analyze processes associated with the service's PID.


5. Remediation: Stopping and Removing Malicious Services

1. Stop the Service

Prevent it from running:

sudo systemctl stop SERVICE_NAME

2. Disable the Service

Prevent it from starting on boot:

sudo systemctl disable SERVICE_NAME

3. Delete the Service File

Remove the service configuration:

sudo rm /lib/systemd/system/SERVICE_NAME.service

4. Reload Daemon to Apply Changes

sudo systemctl daemon-reload

5. Verify Removal

Ensure the service no longer exists:

systemctl list-units --type=service | grep SERVICE_NAME

6. Post-Incident Actions

  1. Review Recent Changes:

    • Use find to list recently modified files:

      find /etc/systemd/system/ -type f -mtime -5
  2. Reinforce Access Controls:

    • Restrict who can create or modify services.

  3. Enable File Integrity Monitoring:

    • Use tools like Tripwire or AIDE to monitor system files for changes.

  4. Regular Audits:

    • Periodically review service configurations and logs for anomalies.


Key Points

Linux services are a common vector for attackers aiming to maintain persistence. By analyzing service configurations, system logs, and active processes, incident responders can uncover and mitigate malicious activities. Proper eradication of malicious services ensures that attackers cannot re-establish control, securing the system against further exploitation.

Last updated