/
Checkmk Agent Troubleshooting Checklist

Checkmk Agent Troubleshooting Checklist

A step-by-step checklist to troubleshoot Checkmk agent issues across Windows, Linux, and the server.

LAST TESTED ON CHECKMK 2.4.0P22

Table of Contents

Overview

This checklist provides a structured approach to troubleshooting Checkmk agent issues across Windows, Linux, and the Checkmk server itself. It walks through the most common failure points such as services, connectivity, runtime performance, registration, and logs, so you can quickly isolate where the problem is occurring and take action.

 

Windows agent

Commands generally should be run on PowerShell with administrative privileges.

 

Service

This section verifies that the Checkmk agent service on Windows is running as expected, since the agent will not respond if the service is stopped or unstable. First, confirm that the service is in a healthy running state. If you are seeing outdated data or suspect caching issues, restart the service, as the Windows agent keeps its cache in memory and a restart clears it.

Check service: Get-Service | Where-Object {$_.Name -like "*cmk*"}
Restart: Restart-Service "Checkmk Service"

 

Connectivity

Focuses on basic network communication between the Windows host and the Checkmk server. This includes verifying required ports and confirming that firewall rules are not blocking traffic.

Note that port 8000 is the default agent port. In Checkmk, the agent controller uses dynamically assigned ports depending on the configuration and connection mode, so this value may differ in your environment. For most setups, you should verify which port is actually in use before testing.

For more details, see the Checkmk Ports article.

Outbound firewall:  Windows Firewall allows outbound connections on ports 8000 and 6556
Inbound Firewall: Windows Firewall has an inbound rule allowing connections on port 6556
Test port: Test-NetConnection -ComputerName <server_ip> -Port 8000
Telnet test: telnet <server_ip> 8000

 

Agent Runtime (must be under 60s)

Validates how long the agent takes to execute locally. If the runtime exceeds 60 seconds, it usually indicates a slow or hanging plugin, which can cause timeouts and missing data.

Run: Measure-Command { & "C:\Program Files (x86)\checkmk\service\check_mk_agent.exe" }

 

Registration

Covers the agent’s registration status with the Checkmk site. If the agent is not properly registered, encrypted communication and push mode will fail.

Check status: & "C:\Program Files (x86)\checkmk\service\cmk-agent-ctl.exe" status
Re-register: & "C:\Program Files (x86)\checkmk\service\cmk-agent-ctl.exe" register --site <site> --user agent_registration --password "<pw>" --server <ip> --hostname <hostname>

 

Agent Updater

Ensures the agent updater is configured and working correctly. Problems here can prevent automatic updates or cause version mismatches.

Agent Updater registration: & "C:\Program Files (x86)\checkmk\service\check_mk_agent.exe" updater register
Agent Updater configuration: & "C:\Program Files (x86)\checkmk\service\check_mk_agent.exe" updater show-config

 

Agent Output

Verifies that the agent is actually producing valid output. Running the agent manually helps confirm whether data collection is working as expected.

Agent output: & "C:\Program Files (x86)\checkmk\service\check_mk_agent.exe" test

 

Example output:

PS C:\Users\vagrant> & "C:\Program Files (x86)\checkmk\service\check_mk_agent.exe" test <<<check_mk>>> Version: 2.4.0p22 BuildDate: AgentOS: windows Hostname: mysite-windows Architecture: 64bit OSName: Microsoft Windows Server 2019 Standard Evaluation OSVersion: 10.0.17763 OSType: windows ... <<<fileinfo:sep(124)>>>

 

Logs

Points to Windows-specific logging locations such as Event Viewer and agent log directories. Logs are critical for identifying errors that are not visible from command output.

Windows server logs: Event Viewer → Windows Logs → Application → filter checkmk
Windows agent logs: C:\ProgramData\checkmk\agent\log\

 

Articles related to common errors:

 

Linux agent

Service

Checks the status of the agent controller and socket on Linux systems. If these services are not running, the agent will not accept connections.

Status: systemctl status cmk-agent-ctl-daemon.service
Status: systemctl status check-mk-agent.socket
Restart: systemctl restart check-mk-agent.socket
Restart: systemctl restart cmk-agent-ctl-daemon.service

 

Example output:

OMD[mysite]:~$ systemctl status check-mk-agent.socket ● check-mk-agent.socket - Local Checkmk agent socket Loaded: loaded (/usr/lib/systemd/system/check-mk-agent.socket; enabled; preset: enabled) Active: active (listening) since Mon 2026-04-27 09:14:00 CEST; 2 days ago Docs: https://docs.checkmk.com/latest/en/agent_linux.html Listen: /run/check-mk-agent.socket (Stream) Accepted: 1900; Connected: 0; Tasks: 0 (limit: 37491) Memory: 0B (peak: 256.0K) CPU: 413us CGroup: /system.slice/check-mk-agent.socket OMD[mysite]:~$ systemctl status cmk-agent-ctl-daemon.service ● cmk-agent-ctl-daemon.service - Checkmk agent controller daemon Loaded: loaded (/usr/lib/systemd/system/cmk-agent-ctl-daemon.service; enabled; preset: enabled) Active: active (running) since Mon 2026-04-27 09:14:13 CEST; 2 days ago Docs: https://docs.checkmk.com/latest/en/agent_linux.html Main PID: 2896 (cmk-agent-ctl) Tasks: 4 (limit: 37491) Memory: 2.3M (peak: 8.8M) CPU: 44.923s CGroup: /system.slice/cmk-agent-ctl-daemon.service └─2896 /usr/bin/cmk-agent-ctl daemon

 

Connectivity & Firewall

Validates that the agent port is listening and accessible. This includes checking firewall rules and ensuring that port 6556 is open and reachable.

Port listening: ss -tlnp | grep 6556
netstat: netstat -tlnp | grep 6556
UFW status: sudo ufw status verbose
Allow port: sudo ufw allow 6556/tcp && sudo ufw reload
iptables: sudo iptables -L INPUT -n | grep 6556

 

Agent Runtime (must be under 60s)

Validates how long the agent takes to execute locally. If the runtime exceeds 60 seconds, it usually indicates a slow or hanging plugin, which can cause timeouts and missing data.

Run locally: time check_mk_agent


Example output:

OMD[mysite]:~$ time check_mk_agent <<<check_mk>>> Version: 2.4.0p22 AgentOS: linux Hostname: mysite AgentDirectory: /etc/check_mk DataDirectory: /var/lib/check_mk_agent SpoolDirectory: /var/lib/check_mk_agent/spool PluginsDirectory: /usr/lib/check_mk_agent/plugins LocalDirectory: /usr/lib/check_mk_agent/local OSType: linux ... real 0m0,285s user 0m0,143s sys 0m0,146s

 

Manual Test

Confirms agent functionality by running it directly on the host or testing connectivity from the server. This helps isolate whether the issue is local or network-related.

Run locally: check_mk_agent
Telnet from server: telnet <agent_ip> 6556
nc test: nc -zv <agent_ip> 6556
Run the agent manually: check_mk_agent
Run the agent with debug: check_mk_agent -d &>debug.log

 

Registration

Ensures the agent is properly registered with the Checkmk server. Misconfigured or missing registration will break secure communication.

Check Status: cmk-agent-ctl status
Re-register: cmk-agent-ctl register --site <site> --user agent_registration --password "<pw>" --server <ip>

 

Logs

Provides access to system logs and agent-specific logs on Linux. These logs help identify service failures, permission issues, or runtime errors.

Linux agent systemd logs: journalctl -u cmk-agent-ctl-daemon.service -f
Linux agent log: /var/log/check_mk_agent/
Linux agent log: /var/lib/check_mk_agent/

 

Articles related to common errors:

 

Checkmk Site (Server Perspective)

Site Health

Verifies that the Checkmk site itself is running and healthy. If the site is down or partially running, agents will not be able to communicate with it.

Site running: omd status <site>
Restart: omd restart <site>
Agent receiver enabled: omd config <site> show | grep AGENT_RECEIVER
Agent receiver port: omd config <site> show | grep AGENT_RECEIVER_PORT

 

Example output:

root@mylinuxhost:~$ omd status mysite jaeger: running agent-receiver: running mkeventd: running liveproxyd: running mknotifyd: running rrdcached: running redis: running automation-helper: running ui-job-scheduler: running cmc: running apache: running dcd: running stunnel: running xinetd: running metric-backend: running otel-collector: running crontab: running --------------------------- Overall state: running root@mylinuxhost:~$ omd config mysite show | grep AGENT_RECEIVER AGENT_RECEIVER: on AGENT_RECEIVER_PORT: 8000 root@mylinuxhost:~$ omd config mysite show | grep AGENT_RECEIVER_PORT AGENT_RECEIVER_PORT: 8000

 

Network & Firewall

Checks that the server is listening on required ports and that no firewall rules are blocking incoming or outgoing connections. This ensures agents can reach the server.

Port 8000 listening: ss -tlnp | grep 8000
Port 6556 listening: ss -tlnp | grep 6556
netstat: netstat -tlnp | grep 8000
Telnet from agent to server: telnet <server_ip> 8000
UFW status: sudo ufw status verbose
Allow if missing: sudo ufw allow 8000/tcp && sudo ufw reload

 

 

Agent Runtime from Server (must be under 60s)

Validates how long the agent takes to execute locally. If the runtime exceeds 60 seconds, it usually indicates a slow or hanging plugin, which can cause timeouts and missing data.

Run locally: time cmk -d <hostname>

 

Example output:

OMD[mysite]:~$ time cmk -d mysite <<<check_mk>>> Version: 2.4.0p22 AgentOS: linux Hostname: mysite AgentDirectory: /etc/check_mk DataDirectory: /var/lib/check_mk_agent SpoolDirectory: /var/lib/check_mk_agent/spool PluginsDirectory: /usr/lib/check_mk_agent/plugins LocalDirectory: /usr/lib/check_mk_agent/local OSType: linux ... real 0m19.947s user 0m3.185s sys 0m0.287s

 

 

Setup & Configuration

Ensures that the host is correctly configured in Checkmk and that all changes have been activated. Misconfigurations here can prevent data from being processed even if the agent is working.

Check Host: Host exists and configured correctly in Setup → Hosts
Check pending changes: Changes have been activated (no pending changes)
Run discovery: cmk -I <hostname>
Apply changes: cmk -R

 

Logs (site user)

Highlights the most important server-side log files. These logs provide visibility into agent communication, errors, and processing issues from the Checkmk core.

Access log: tail -f ~/var/log/agent-receiver/access.log
Error log: tail -f ~/var/log/agent-receiver/error.log
CMC log: tail -f ~/var/log/cmc.log
Web log: tail -f ~/var/log/web.log

 

 

Related articles