Troubleshooting Apache service not starting
This article details how to troubleshoot apace not starting on a Checkmk server.
LAST TESTED ON CHECKMK 2.0.0P1
We do not support any modification of our config files (e.g., apache, .profile, etc.). If you do that and the site is broken, you will run out of support, especially on the Checkmk Appliance.
We saw the bottom error several times, especially when dealing with check_sql.
Error Messages
Starting apache...apache2: Syntax error on line 236 of /omd/sites/mysite/etc/apache/apache.conf libpython3.9.so.1.0: cannot open shared object file: No such file or directory
Problem
The default for LD_LIBARY_PATH should be:
LD_LIBRARY_PATH=$OMD_ROOT/local/lib:$OMD_ROOT/lib
But for some reason, the .profile file of the site user seems to be broken, and after the restart of the site, you will face the following issue:
Starting apache...apache2: Syntax error on line 236 of /omd/sites/mysite/etc/apache/apache.conf: Syntax error on line 2 of /omd/sites/mysite/etc/apache/conf.d/01_wsgi.conf: Cannot load /omd/sites/mysite/lib/apache/modules/mod_wsgi.so into server: libpython3.9.so.1.0: cannot open shared object file: No such file or directory ..........ERROR
Solution
Check if the env is right
su - mysite OMD[mysite]:~$ echo $LD_LIBRARY_PATH /omd/sites/mysite/opt/oracle/instantclient_21_4
In that case, the env is not correct. The env is pointing only to the libraries of the Oracle instantclient
Check if the library is available.
OMD[mysite]:~$ find -L -name "*libpython3.9.so.1.0*" ./version/lib/libpython3.9.so.1.0 ./lib/libpython3.9.so.1.0
In that case, the library is only available in the origin and default path of LD_LIBRARY_PATH
Restore the default variable
You will find the original files of .profile or .bashrc in "/omd/versions/2.1.0b7.cee/skel"
➜ /omd/versions/2.1.0b7.cee/skel ls -altr insgesamt 36 -rw-r--r-- 1 root root 2055 Apr 25 00:02 .profile -rw-r--r-- 1 root root 1091 Apr 25 00:02 .bashrc
One way would be to copy the LD_LIBRARY_PATH value from .profile and put it directly to the .profile of the site user:
su - mysite OMD[mysite]:~$ vi .profile
To make the changes permanent, please enter:OMD[mysite]:~$ source .profile
set the LD_LIBARY_PATH the right way
extend the LD_LIBRARY_PATH like that ":path/to/oracle/instanclient"
OMD[mysite]:~$ vi .profile OMD[mysite]:~$ source .profile
In the end, it should look like this:OMD[mysite]:~$ echo $LD_LIBRARY_PATH /omd/sites/mysite/local/lib:/omd/sites/mysite/lib:path/to/oracle/instanclient
- Now after a restart of the site Apache the error should be gone!
Related articles