Troubleshooting antivirus and security software interference with Checkmk

Troubleshooting antivirus and security software interference with Checkmk

This article explains how to identify security software interference with Checkmk and configure exclusions to prevent it.

LAST TESTED ON CHECKMK 2.5.0P14

Warning

This article provides guidance on reducing possible interference from security software with Checkmk. However, we cannot guarantee that these steps will always be effective, as modern security solutions can use mechanisms that are difficult to track or understand.

Checkmk does not provide support for antivirus or Endpoint Detection and Response (EDR) software. Any assistance related to third-party security software is provided on a best-effort basis.

 

Identify signs of security software interference

Security software can interfere with network traffic, processes, files, binaries, and libraries used by Checkmk. The symptoms generally fall into three categories.

 

TLS connectivity issues

Some security software modifies the operating system trust store or intercepts TLS-encrypted network traffic. In Checkmk, this can interfere with agent registration, which uses mutual TLS (mTLS) between the Checkmk agent and site server.

Common signs of TLS interference include:

  • Certificates are presented that were issued by the security software vendor.

  • TLS handshake failures or connection resets occur, while non-TLS traffic between the same endpoints works normally.

  • Certificate fingerprints differ from the expected values.

 

Unexpected permission and execution issues

Checkmk uses various files, binaries, and libraries, including components provided by the operating system. Security software can use heuristics or other detection mechanisms to monitor processes and deny access to these resources.

Common signs include:

  • Checkmk receives permission errors when running non-interactively, while the same commands work when executed interactively as the same user.

  • Permission denied errors occur even though the operating system ACLs permit access.

  • An error reports that a file does not exist even though the file is present on the filesystem.

  • Common system binaries such as zip or tar, or common Windows DLLs, become inaccessible.

  • Binaries crash with unexpected error codes, even when using simple arguments such as --help or --version.

 

Files disappear or change unexpectedly

Security software can also interfere directly with files created or modified by Checkmk. These issues can be particularly difficult to diagnose because the filesystem can appear to behave inconsistently.

Common signs include:

  • Files disappear after being saved or after a consistent period of time.

  • Changes to files do not persist after saving.

  • Operating system tools report that a file was never modified even though changes were made.

  • The contents of a file differ depending on the context in which the file is viewed.

 

Confirm whether security software is the cause

If you encounter one or more of these symptoms, temporarily disable the security software on both the Checkmk site server and the affected monitored host, where possible.

Repeat the operation that previously failed. If the problem no longer occurs while the security software is disabled, this is a strong indication that the security software is interfering with Checkmk.

If disabling the security software is not possible, review its event, detection, quarantine, and audit logs for activity involving Checkmk processes or files.

 

Confirm antivirus exclusions

To reduce the likelihood of security software interfering with Checkmk, exclude the relevant Checkmk directories from scanning.

Depending on how your antivirus or EDR solution operates, directory exclusions alone might not prevent all interference. Security software can also inspect processes, network connections, libraries, and other system activity.

 

Checkmk server

Exclude the following directories on the Checkmk server:

/opt/omd/apache/ /opt/omd/sites/ /opt/omd/versions/ /omd/

These are the server-side exclusions specified for Checkmk.

 

Checkmk Windows agent

Exclude the following directories on monitored Windows hosts:

C:\Program Files (x86)\checkmk\ C:\ProgramData\checkmk\

These directories contain files associated with the Checkmk Windows agent.

For a detailed listing of the relevant files and directories on monitored hosts, see Paths on the monitored host.

 

Robotmk and Microsoft Defender

If you use Robotmk together with Microsoft Defender, additional exclusions might be required for Robotmk binaries.

See Excluding Robotmk binaries from Microsoft Defender for the applicable configuration.

 

Collect diagnostic information

If the problem continues after configuring exclusions, collect diagnostic information from both the Checkmk server and the affected monitored host.

A Checkmk support diagnostic from the site server and an agent output from the affected monitored host are useful starting points. These can help identify processes running on the systems that might be interfering with Checkmk.

Collect the process list on Linux

Run the following command as root:

ps aux > output.txt

 

Collect the process list on Windows

Run the following PowerShell command as an administrator:

Get-Process | Out-File output.txt

Review the process list for antivirus, EDR, or other security software running on the affected system.

 

Note

Running security software directly on the Checkmk Appliance is unsupported.

 

Related articles