...
| Warning |
|---|
Before you delve into low-level debugging of why the CMC is running but not working (without a stack trace), please check the "Master Control" snap-in in the sidebar first! If the Service Checks and Host Checks are disabled, that might be the reason for your problem.
|
Analyze CMC core
strace
You can use strace to track the CMC process when you face any issue:
| Code Block | ||||
|---|---|---|---|---|
| ||||
root@linux~# strace -o cmc-strace.log -p $(cat ~<MYSITE>/tmp/run/cmc.pid) |
valgrind
You can use valgrind to start the CMC in the debug mode. Here you will get a full stack trace. If valgrind is unavailable on your system, install it or run the CMC only with the -g option.
| Code Block | ||||
|---|---|---|---|---|
| ||||
root@linux~# su mysite OMD[mysite]:~$ omd stop cmc OMD[mysite]:~$ valgrind --num-callers=30 cmc -g or root@linux~# su mysite OMD[mysite]:~$ omd stop cmc OMD[mysite]:~$ cmc -g |
gdb
With gdb, you can analyze the coredump if checkmk will create one. Note: Checkmk will only create one if you enable it in the global settings.
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
root@linux:~# gdb /omd/sites/mysite/bin/cmc --core=<PATH/TO/COREUMP> (gdb) r |
frozen CMC
When the CMC seems to freeze and nothing happens, please run this command before restarting the CMC:
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
root@linux:~# for iter in {1..60}; do
printf "\nrun %i\n\n" $iter
gdb -p "$(cat "/omd/sites/mysite/tmp/run/cmc.pid")" --batch -ex 'set pagination off' -ex 'thread apply all backtrace' || true
sleep 5
done |& tee /home/mylinuxuser/Downloads/gdb.txt |
Analyze coredump file
| Note |
|---|
By default, there is no coredump creation enabled. You can enable that via Setup → Global settings → Monitoring core → Enable core dumps After a crash of the CMC, a coredump in ~/var/check_mk/core/ will be written |
gdb
With gdb, you can analyze the coredump if checkmk will create one. Note: Checkmk will only create one if you enable it in the global settings.
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
set logging file gdb_log.txt set logging on set trace-commands on show logging # prove logging is on flush set pretty print on bt # view the backtrace set logging off show logging # prove logging is back off |
objdump
With objdump, you can fetch the content of the dump.
| Code Block | ||||
|---|---|---|---|---|
| ||||
root@linux:~# objdump -s /mypath_tofile/core.python3.989.4b7ee3adffd14e31a0188aac0c215161.804036.1640164046000000 >dump_sup8890.txt |
file command
With the file command, you can also fetch the content of the dump.
| Code Block | ||||
|---|---|---|---|---|
| ||||
# Command: file /mypath_tofile/core.python3.989.4b7ee3adffd14e31a0188aac0c215161.804036.1640164046000000 # Output: /mypath_tofile/core.python3.989.4b7ee3adffd14e31a0188aac0c215161.804036.1640164046000000: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'python3 /omd/sites/mysite/bin/cmk --discover-marked-hosts', real uid: 989, effective uid: 989, real gid: 1000, effective gid: 1000, execfn: '/omd/sites/mysite/bin/python3', platform: 'x86_64' |
Open a support case
If your investigation is not successful, please open a ticket and provide us with the following data:
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
Please send us the following data to help us reproduce the issue.
* Login as a site user with {{su - $MYSITE}} and
* create an archive with the following command {{tar czf ~/corefiles.tgz ~/var/check_mk/core/ ~/var/log/}}. |
Useful links
Related articles
| Filter by label (Content by label) | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...