Troubleshooting - Cannot activate changes. Another activation process is currently in progress or locked

Owned by Former user (Deleted)
Last updated: Sept 25, 2024 by Matthew Hierholzer
1 min read

There are different error messages, to which this article may apply. Read the Problem section, to learn which exact error messages are relevant.

LAST TESTED ON CHECKMK 2.2.0P1

Table of Contents

Problem

You may face ones of these problems while activating changes. The exact error message might look different, but this article might still apply.


 

One of these error messages might apply to you:

Progress: Failed
Got invalid data:

Internal automation error: Error running automation call restart (exit code 1), error:

An error occurred: Other restart currently in progress. Aborting

Traceback (most recent call last):
File "/omd/sites/mysite/lib/python3/cmk/gui/wato/pages/automation.py", line 231, in
_execute_automation_command
response.set_data(repr(automation.execute(automation.get_request())))

File "/omd/sites/mysite/lib/python3/cmk/gui/wato/pages/activate_changes.py", line 1034, in
execute
return activate_changes.execute_activate_changes(


File "/omd/sites/mysite/lib/python3/cmk/gui/watolib/activate_changes.py", line 2128. in
execute_activate_changes

warnings - get_config_domain(domain_request name)().activate(domain_request.settings)

File "/omd/sites/mysite/lib/python3/cmk/gui/watolib/config_domains.py", line 76, in activate
return ("restart": restart, "reload": reload,)[

File "/omd/sites/mysite/lib/python3/cmk/gui/watolib/check_mk_automations.py*. line 275, in
restart
automation_serialized("restart", args=hosts_to_update)

File "/omd/sites/mysite/lib/python3/cmk/gui/watolib/check_mk_automations.py", line 53. in automation_serialized
cmdline, serialized_result check_mk_local_automation_serialized(

File "/omd/sites/mysite/lib/python3/cmk/gui/watolib/automations.py". line 148. in check_mk_local_automation_serialized raise local_automation_failure(
cmk.gui.watolib.automations.MKAutomationException: Error running automation call restart (exit
code 1), error

An error occurred Other restart currently in progress, Aborting
Progress: Locked
The site is currently locked by another activation process. 

Please try again later.

Verification

  1. Log into your central Checkmk site as an administrative user and navigate to Background jobs.
  2. Check the section titles Activate Changes Scheduler for running jobs, which are older than a few minutes.
  3. Take note of the UUID at the end of the job name (activate-changes-scheduler-263383ff-b58b-4a8e-ba77-c6e1ff2d3b94).

If you see one or more of the aforementioned jobs, move forward with this article.

Solution A (recommended)

As you verified in the section above, there are lingering files within the central site's ~/tmp/check_mk/wato/activation/ directory. 

  1. A reboot of your Checkmk server's operating system will clear out these files and should allow you to activate changes again.

Solution B

If for whatever reason you cannot or do not want to reboot the server, you can delete the files manually on the command line. We recommend this only be done by experienced users.


  1. Login to your central site as the site user

    root@linux:~# omd su mysite


    .

  2. Check to see if you have old directories within $OMD_ROOT/tmp/check_mk/wato/activation/

    OMD[mysite]:~$ ls -lah $OMD_ROOT/tmp/check_mk/wato/activation/
total 42

drwxrwx--- 3 mysite mysite 100 Jul 4 00:42 6ecd4860-611e-4a4c-912f-74a6f1123e99/
drwxrwx--- 3 mysite mysite 100 Jul 4 00:42 bec06bbc-1d35-4668-a421-151828b8500d/
drwxrwx--- 3 mysite mysite 920 Jul 3 18:34 263383ff-b58b-4a8e-ba77-c6e1ff2d3b94/

    It looks like there is one old directory that couldn't be deleted. The files here are deleted every minute by default.


    .

  3. Remove the lingering directory:

    OMD[mysite]:~$ rm -r $OMD_ROOT/tmp/check_mk/wato/activation/263383ff-b58b-4a8e-ba77-c6e1ff2d3b94/


    .

  4. There will be a corresponding background job in $OMD_ROOT/var/check_mk/background_jobs/, which can be cleaned up as well:

    OMD[mysite]:~$ ls -lah $OMD_ROOT/var/check_mk/background_jobs/
total 42
drwx------  2 mysite mysite 4.0K Jul 4 00:42 activate-changes-scheduler-0897307d-d177-4b6e-a500-1263a5ec3299/
drwx------  2 mysite mysite 4.0K Jul 4 00:42 activate-changes-scheduler-1ba662c2-374a-4bce-9b6a-59329e9f11a5/
drwx------  2 mysite mysite 4.0K Jul 3 18:34 activate-changes-scheduler-263383ff-b58b-4a8e-ba77-c6e1ff2d3b94/

OMD[mysite]:~$ rm -R $OMD_ROOT/var/check_mk/background_jobs/activate-changes-scheduler-263383ff-b58b-4a8e-ba77-c6e1ff2d3b94/


    .

  5. Now go back to the UI and try to activate changes.


Related articles