Checkmk profiling

LAST TESTED ON CHECKMK 2.2.0P1

Step-by-step guide

GUI Profiling

.

Enable profiling in Checkmk

1. Setup → General → Global Settings → User Interface → Profile Request
   
   .
   

2. Modify the URL to profile by adding '&_profile=1' and fire the request in your browser

   http://localhost/nagnis_master/check_mk/view.py?view_name=allhosts&_profile=

   .

3. Now two files are created in ~/var/check_mk/

   root@mylinuxhost:/opt/omd/sites/mysite/var/check_mk# ll |grep multisite
   -rw-rw---- 1 mysite mysite 100246 Nov 6 13:35 multisite.profile
   -rwxr-xr-x 1 mysite mysite 155 Nov 6 13:35 multisite.py*

   .

4. By executing 'multisite.py', you can get runtime statistics about the last processed page.

   root@mylinuxhost:/opt/omd/sites/mysite/var/check_mk# ./multisite.py |more
   Fri Nov  6 13:35:33 2020    /omd/sites/nagnis_master/var/check_mk/multisite.profile
   
            9585 function calls (9480 primitive calls) in 0.007 seconds
   
      Ordered by: internal time
   
      ncalls  tottime  percall  cumtime  percall filename:lineno(function)
           1    0.001    0.001    0.002    0.002 /omd/sites/mysite/lib/python/cmk/gui/modules.py:140(_cmk_gui_top_level_modules)
        4652    0.000    0.000    0.000    0.000 {method 'startswith' of 'str' objects}
   ...
   ...

   .

5. For viewing multisite. Profile with snakeviz, move the file to another directory (e.g., /tmp), and change the owner to the owner of your Internet Browser.

   linuxuser@mylinuxhost:/omd/sites/mysite/var/check_mk$ sudo cp multisite.profile /tmp/
   linuxuser@mylinuxhost:/tmp$ sudo chown linuxuser:linuxuser multisite.profile
   

   .

6. Install the Python module (for Python 2 or Python 3, respectively)

   root@mylinuxhost:~# pip install snakeviz
   root@mylinuxhost:~# pip3 install snakeviz

   .

7. Now you can use snakeviz to view the profiling file.

   linuxuser@mylinuxhost:/tmp$ snakeviz multisite.profile

   
   

.Now you can analyze at which point of the request your view needs the most time!

Profiling via cli

1. The cmk commands have the option to profile them.

   OMD[mysite]:~$ cmk |grep profile
      --profile           Enable profiling mode

   .

2. The profile file will be created in the current directory. Let's go to tmp and run the cmk command with the profile option.

   ➜  ~ su - at                                        
   OMD[mysite]:~$ cd tmp
   OMD[mysite]:~$ cmk -Afv --profile

   cmk -Afv --profile is only one example. This command will trigger the agent baking via the CLI. You may need to profile a different command.

   .

   • 1. Now we will see two files:

        OMD[mysite]:~/tmp$ ls |grep profile
        profile.out*
        show_profile.py*

        .

     2. To work with these files, we need to copy them outside the site.

        OMD[mysite]:~/tmp$ cp show_profile.py profile.out /tmp/

        .

     3. Now there are two ways to analyze these files:
        
        

        1. Open the profile file via the command line:

           ➜  /tmp ./show_profile.py 

        2. Analyze via snakevize as described here: Enable Checkmk profiling#GUIProfiling
           
           

Profile a Checkmk function

There is an option to profile a function inside Checkmk. This can be either your own code, e.g., a plugin or a checkmk-specific code:

from cmk.utils.profile import profile_call
.....

@profile_call("/tmp/myprofile_dir")
def my_func_to_profile(self):
    print("ding dong")

Analyze a profiling file with python3 but generated on python2

There is a simple to achieve this anyway:

The solution is: Python virtualenv inspired by: https://computingforgeeks.com/how-to-install-python2-with-virtualenv-on-ubuntu/

Here is my short step-by-step guide:

# Install virtualenv
sudo apt install virtualenv

# Create a directory for virtualenv
mkdir ~/.virtualenvs

# Change to this directory and create a virtualenv with the python interpreter you want (we will use python2)
cd ~/.virtualenvs
virtualenv --python=python2 env

# Confirm that the right python version is installed
ls env/lib
 
# Activate the new virtualenvironment
source env/bin/activate

# Now you can install all kind of modules inside this python2 context
pip install snakeviz

In this env, you can analyze all profile files generated with python2!

Network Analyze with the Internet browser

1. Start the Network Monitor
   • Ctrl + Shift + I
     
   • For Firefox, follow this manual: https://developer.mozilla.org/en-US/docs/Tools/Network_Monitor
   • For Google Chrome, follow this manual: https://developer.chrome.com/docs/devtools/network/
     
     
2. Reload the slow/ crashing view again and save/export this file as .har. Please submit this file to us.

Slow views

Some built-in or own views may take longer time than expected. In order to detect slow views, you have to set:

• The log level to DEBUG at Setup > General > Global settings > User interface > Log levels > Slow views

• A threshold (in seconds) at Setup > General > Global settings > User interface > Threshold for slow views
  
  

The logging is disabled by default. The default threshold is set to 60 seconds.

If enabled, one log entry per view rendering that exceeds the configured threshold is logged to var/log/web.log.

A slow-view log entry will look like the following:

2023-10-05 14:01:02,467 [10] [cmk.web.slow-views 783147] View name: logfile, User: cmkadmin, Row limit: None, Limit type: none, URL variables: ['csrf_token=deaacdae-e2ad-47bb-a497-8a41c4f5e378', 
'filled_in=filter', 'is_log_notification_phase=-1', 'limit=none', 'logclass0=on', 'logclass1=on', 'logclass2=on', 'logclass3=on', 'logclass5=on', 'logclass8=on', 'logst_h0=on', 'logst_h1=on', 
'logst_h2=on', 'logst_s0=on', 'logst_s1=on', 'logst_s2=on', 'logst_s3=on', 'logtime_from=400', 'logtime_from_range=86400', 'logtime_until_range=3600', 'view_name=logfile', '_show_filter_form=1'], 
View context: {'optservicegroup': {'optservice_group': '', 'neg_optservice_group': ''}, 'siteopt': {'site': ''}, 'log_class': {'logclass0': 'on', 'logclass1': 'on', 'logclass2': 'on', 'logclass3': 
'on', 'logclass4': '', 'logclass5': 'on', 'logclass6': '', 'logclass8': 'on'}, 'log_notification_phase': {'is_log_notification_phase': '-1'}, 'hostregex': {'host_regex': ''}, 'serviceregex': 
{'service_regex': ''}, 'opthostgroup': {'opthost_group': '', 'neg_opthost_group': ''}, 'logtime': {'logtime_from': '400', 'logtime_until': '', 'logtime_from_range': '86400', 'logtime_until_range': '3600'},
'log_state': {'logst_h0': 'on', 'logst_h1': 'on', 'logst_h2': 'on', 'logst_s0': 'on', 'logst_s1': 'on', 'logst_s2': 'on', 'logst_s3': 'on'}, 'wato_folder': {'wato_folder': ''}}, Unfiltered rows: 
24783, Filtered rows: 24783, Rows after limit: 0, Duration fetching rows: 0.69s, Duration filtering rows: 0.00s, Duration rendering view: 18.44s, Rendering page exceeds 3s: 19.21s

Related articles

• Page:
  Enable Checkmk profiling
• Page:
  Graphical User Interface (GUI)
• Page:
  How-to enable Pretty-print configuration files
• Page:
  How-to work with HTTP(s) Iframes in Checkmk
• Page:
  How-to rename fs_ Service to Filesystem