Troubleshooting pending changes error

Owned by Anastasios Thomaidis
Last updated: Nov 20, 2024 by Matthew Hierholzer

This article explains how to troubleshoot a pending activation error while working with a remote site.

LAST TESTED ON CHECKMK 2.3.0P1

Table of Contents

Problem

During the activation of changes from the central site for a remote site, we faced an issue. This will happen if WATO is enabled on the remote site and you have pending changes there.

Failed: Garbled automation reponse
Internal automation error: There are 1 pending changes that would get lost. The most recent are: Modified host p19central-vcenter-agent.txt.

Traceback (most recent call last):

File "/omd/sites/remote/lib/python/cmk/gui/wato/pages/automation.py".line 186, in execute_automation _command
html.write(repr(automation.execute(automation.get_request()))File &quot:/omd/sites/remote/lib/python/cmk/gui/watolib/sites.py&quot:. line 717, in get
request self._verify_remote_site_config(site_id) File &quot/omd/sites/remote/lib/python/cmk/gui/watolib/automation_commands.pyquot:, line 63, in
verify_remote_site_config raise MKGeneralException(message) MKGeneralException: There are 1 pending changes that would get lost. The most recent are:
Modified host p19central-vcenter-agent.txt

On the remote:

On the remote site, the pending change is stored in the ~/var/check_mk/wato/replication_changes_remote.mk file.

OMD[remote]:~/var/check_mk/wato$ ll
total 36
drwxrwx--- 2 remote remote 4096 Dec 11 10:07 activation/
drwxrwxr-x 2 remote remote 4096 Dec 11 13:24 auth/
-rw-rw---- 1 remote remote 35 Dec 10 14:14 automation_secret.mk
-rw-rw---- 1 remote remote 9 Dec 10 14:12 last_bake.mk
drwxrwxr-x 2 remote remote 4096 Dec 10 14:12 log/
drwxrwx--- 2 remote remote 4096 Dec 10 14:12 php-api/
-rw-rw---- 1 remote remote 301 Dec 11 13:24 replication_changes_remote.mk
-rw-rw---- 1 remote remote 130 Dec 11 12:00 replication_status_remote.mk
drwxrwxr-x 3 remote remote 4096 Dec 11 12:00 snapshots/


~/var/check_mk/wato
OMD[remote]:~/var/check_mk/wato$ cat replication_status_remote.mk 
{'current_activation': None, 'last_activation': 'f9607439-2303-412c-a188-f4e0f54702ae', 'times': {'restart': 1.6235217094421388}}
OMD[remote]:~/var/check_mk/wato$ cat replication_changes_remote.mk 
{'need_restart': True, 'need_sync': True, 'user_id': u'cmkadmin', 'time': 1607689496.047742, 'domains': ['check_mk'], 'text': u'Modified host p19central-vcenter-agent.txt.', 'object': ('CREHost', 'p19central-vcenter-agent.txt'), 'id': 'cc91dcc8-bcc6-46fa-a264-9bdc5b041720', 'action_name': 'edit-host'}

Solution

In order to activate changes successfully from central to remote, you need to delete both replication-Files on the remote:

OMD[remote]:~/var/check_mk/wato$ rm replication_*

To avoid this issue, disable WATO on the remote and push the configuration only from central.