Page Comparison

...

Code Block

language	bash
theme	RDark

OMD[mysite]:~$ cmk -v --convert-rrds --delete-rrds RRD8040
RRD8040:
  HOST: RRD8040
      ERROR: 'NoneType' object has no attribute 'text'

How to debug

...

Info

Let's modify the command to receive more debug output:

Code Block

language	bash
theme	RDark
collapse	true

OMD[mysite]:~$ cmk -vv --debug --convert-rrds --delete-rrds RRD8040
RRD8040:
Traceback (most recent call last):
  File "/omd/sites/rrd/bin/cmk", line 94, in <module>
    exit_status = modes.call(mode_name, mode_args, opts, args)
  File "/omd/sites/rrd/lib/python/cmk_base/modes/__init__.py", line 72, in call
    return mode.handler_function(*handler_args)
  File "/omd/sites/rrd/lib/python/cmk_base/modes/cee.py", line 147, in mode_convert_rrds
    rrd.do_convert_rrds(*args)
  File "/omd/sites/rrd/lib/python/cmk_base/cee/rrd.py", line 413, in do_convert_rrds
    convert_rrds_of_host(hostname)
  File "/omd/sites/rrd/lib/python/cmk_base/cee/rrd.py", line 420, in convert_rrds_of_host
    existing_rrds = find_host_rrd_services(hostname)
  File "/omd/sites/rrd/lib/python/cmk_base/cee/rrd.py", line 711, in find_host_rrd_services
    for service in find_pnp_rrds(hostname):
  File "/omd/sites/rrd/lib/python/cmk_base/cee/rrd.py", line 728, in find_pnp_rrds
    xmlinfo = parse_pnp_xml_file(host_dir + "/" + xml_file)
  File "/omd/sites/rrd/lib/python/cmk_base/cee/rrd.py", line 769, in parse_pnp_xml_file
    'host': root.find('NAGIOS_AUTH_HOSTNAME').text,
AttributeError: 'NoneType' object has no attribute 'text'

Info
It looks like we have some xml files without "NAGIOS_AUTH_HOSTNAME"

To find the corrupted files, you can grep for "NAGIOS_AUTH_HOSTNAME" , save the result to a list and compare this list with all xml files

Code Block

language	bash
theme	RDark
collapse	true

OMD[mysite]:~$ find ~/var/pnp4nagios/perfdata -name "*.xml" | while read xmlfile; do grep "NAGIOS_AUTH_HOSTNAME" $xmlfile >/dev/null || ls $xmlfile; done
/omd/sites/mysite/var/pnp4nagios/perfdata/DB-P02/CPU_utilization.xml
/omd/sites/mysite/var/pnp4nagios/perfdata/DB-P02/Interface_1.xml
/omd/sites/mysite/var/pnp4nagios/perfdata/DB-P-CLUSTER/MSSQL_SQLServer_Locks_Key_Locks.xml
/omd/sites/mysite/var/pnp4nagios/perfdata/DB-P-CLUSTER/MSSQL_SQLServer_Locks_Extent_Locks.xml
/omd/sites/mysite/var/pnp4nagios/perfdata/DB-P-CLUSTER/MSSQL_SQLServer__Total_Transactions.xml
/omd/sites/mysite/var/pnp4nagios/perfdata/DB-P-CLUSTER/MSSQL_SQLServer_Locks__Total_Locks.xml
/omd/sites/mysite/var/pnp4nagios/perfdata/PROXY/Disk_IO_SUMMARY.xml
/omd/sites/mysite/var/pnp4nagios/perfdata/PROXY/CPU_utilization.xml
/omd/sites/mysite/var/pnp4nagios/perfdata/WINDC/Memory_and_pagefile.xml
/omd/sites/mysite/var/pnp4nagios/perfdata/WINDC/Filesystem_E__.xml
OMD[mysite]:~$

If I check each of these files, the files are looking really different. So either the files are corrupt, or they are not in the right format:

Code Block

language	bash
theme	RDark
collapse	true

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE rrd SYSTEM "http://oss.oetiker.ch/rrdtool/rrdtool.dtd">
<!-- Round Robin Database Dump -->
<rrd>
	<version>0003</version>
	<step>60</step> <!-- Seconds -->
	<lastupdate>1497008778</lastupdate> <!-- 2017-06-09 13:46:18 CEST -->

	<ds>
		<name> 1 </name>
		<type> GAUGE </type>
		<minimal_heartbeat>8460</minimal_heartbeat>
		<min>NaN</min>
		<max>NaN</max>

		<!-- PDP Status -->
		<last_ds>5694.328125</last_ds>
		<value>1.0249790625e+05</value>
		<unknown_sec> 0 </unknown_sec>
	</ds>

	<!-- Round Robin Archives -->
	<rra>
		<cf>AVERAGE</cf>
		<pdp_per_row>1</pdp_per_row> <!-- 60 seconds -->

		<params>
		<xff>5.0000000000e-01</xff>
		</params>
		<cdp_prep>
			<ds>
			<primary_value>5.6943281250e+03</primary_value>
			<secondary_value>5.6943281250e+03</secondary_value>
			<value>NaN</value>
			<unknown_datapoints>0</unknown_datapoints>
			</ds>
		</cdp_prep>
		<database>
			<!-- 2017-06-07 13:47:00 CEST / 1496836020 --> <row><v>5.6934375000e+03</v></row>
			<!-- 2017-06-07 13:48:00 CEST / 1496836080 --> <row><v>5.6934375000e+03</v></row>
			<!-- 2017-06-07 13:49:00 CEST / 1496836140 --> <row><v>5.6934375000e+03</v></row>
			<!-- 2017-06-07 13:50:00 CEST / 1496836200 --> <row><v>5.6934375000e+03</v></row>
			<!-- 2017-06-07 13:51:00 CEST / 1496836260 --> <row><v>5.6934375000e+03</v></row>
			<!-- 2017-06-07 13:52:00 CEST / 1496836320 --> <row><v>5.6934375000e

Probably, the filesystem ran full during the conversion. The only solution is to delete these files and run the conversion again!

...

Code Block

language	bash
theme	RDark

OMD[mysite]:~$ cmk --convert-rrds localhost
Traceback (most recent call last):
  File "/omd/sites/rrd/bin/cmk", line 94, in <module>
    exit_status = modes.call(mode_name, mode_args, opts, args)
  File "/omd/sites/rrd/lib/python/cmk_base/modes/__init__.py", line 72, in call
    return mode.handler_function(*handler_args)
  File "/omd/sites/rrd/lib/python/cmk_base/modes/cee.py", line 147, in mode_convert_rrds
    rrd.do_convert_rrds(*args)
  File "/omd/sites/rrd/lib/python/cmk_base/cee/rrd.py", line 413, in do_convert_rrds
    convert_rrds_of_host(hostname)
  File "/omd/sites/rrd/lib/python/cmk_base/cee/rrd.py", line 428, in convert_rrds_of_host
    console.error("      %sERROR: %s%s\n" % (tty.red + tty.bold, e, tty.normal))
UnicodeEncodeError: 'ascii' codec can't encode characters in position 70-75: ordinal not in range(128)

How to debug

...

Info

Let's modify the command to receive more debug output:

Code Block

language	bash
theme	RDark
collapse	true

OMD[rrd]:~$ cmk -vv --debug --convert-rrds localhost
...
  Urlaubstüüüüüüge (PNP)...
    - Metric1..Traceback (most recent call last):
  File "/omd/sites/rrd/bin/cmk", line 94, in <module>
    exit_status = modes.call(mode_name, mode_args, opts, args)
  File "/omd/sites/rrd/lib/python/cmk_base/modes/__init__.py", line 72, in call
    return mode.handler_function(*handler_args)
  File "/omd/sites/rrd/lib/python/cmk_base/modes/cee.py", line 147, in mode_convert_rrds
    rrd.do_convert_rrds(*args)
  File "/omd/sites/rrd/lib/python/cmk_base/cee/rrd.py", line 413, in do_convert_rrds
    convert_rrds_of_host(hostname)
  File "/omd/sites/rrd/lib/python/cmk_base/cee/rrd.py", line 422, in convert_rrds_of_host
    convert_pnp_rrds(hostname, existing_rrds)
  File "/omd/sites/rrd/lib/python/cmk_base/cee/rrd.py", line 441, in convert_pnp_rrds
    cmk.utils.pnp_cleanup(servicedesc), target_rrdconf)
  File "/omd/sites/rrd/lib/python/cmk_base/cee/rrd.py", line 472, in convert_pnp_rrds_of
    result = convert_pnp_rrd(old_rrd_path, new_rrd_path, old_ds_name, rrdconf)
  File "/omd/sites/rrd/lib/python/cmk_base/cee/rrd.py", line 497, in convert_pnp_rrd
    old_rrdconf = get_old_rrd_config(old_rrd_path, old_ds_name)
  File "/omd/sites/rrd/lib/python/cmk_base/cee/rrd.py", line 789, in get_old_rrd_config
    old_config_raw = rrdtool.info([rrd_file_path])
TypeError: Element 0 in argument 0 must be str

Note

And here we go: The issue are the "ü" characters in the service name. This only affects Checkmk 1.6. With Checkmk >2.0 we changed the behavior of non-ASCII Services names (Werk #13711)

Solutions:

If it's related to a builtin Check → please open a ticket via the customer portal
If it's a customization → You need to change the service description of the service, do a rediscovery and move the old pnp4nagios files to the new ones
The easiest way is to con

...

Code Block

language	bash
theme	RDark

OMD[cmk]:~$ cmk --debug -vv --convert-rrds RRD-a
RRD-a:
Traceback (most recent call last):
  File "/omd/sites/cmk/bin/cmk", line 94, in <module>
    exit_status = modes.call(mode_name, mode_args, opts, args)
  File "/omd/sites/cmk/lib/python/cmk_base/modes/__init__.py", line 72, in call
    return mode.handler_function(*handler_args)
  File "/omd/sites/cmk/lib/python/cmk_base/modes/cee.py", line 147, in mode_convert_rrds
    rrd.do_convert_rrds(*args)
  File "/omd/sites/cmk/lib/python/cmk_base/cee/rrd.py", line 413, in do_convert_rrds
    convert_rrds_of_host(hostname)
  File "/omd/sites/cmk/lib/python/cmk_base/cee/rrd.py", line 420, in convert_rrds_of_host
    existing_rrds = find_host_rrd_services(hostname)
  File "/omd/sites/cmk/lib/python/cmk_base/cee/rrd.py", line 711, in find_host_rrd_services
    for service in find_pnp_rrds(hostname):
  File "/omd/sites/cmk/lib/python/cmk_base/cee/rrd.py", line 729, in find_pnp_rrds
    xmlinfo = parse_pnp_xml_file(host_dir + "/" + xml_file)
  File "/omd/sites/cmk/lib/python/cmk_base/cee/rrd.py", line 760, in parse_pnp_xml_file
    root = ET.parse(xml_path).getroot()
  File "/omd/sites/cmk/lib/python2.7/xml/etree/ElementTree.py", line 1182, in parse
    tree.parse(source, parser)
  File "/omd/sites/cmk/lib/python2.7/xml/etree/ElementTree.py", line 656, in parse
    parser.feed(data)
  File "/omd/sites/cmk/lib/python2.7/xml/etree/ElementTree.py", line 1659, in feed
    self._raiseerror(v)
  File "/omd/sites/cmk/lib/python2.7/xml/etree/ElementTree.py", line 1523, in _raiseerror
    raise err
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 5, column 91

How to debug

...

Info

Code Block

language	bash
theme	RDark

vi /omd/sites/cmk/lib/python/cmk_base/cee/rrd.py

...

Code Block

language	bash
theme	RDark
collapse	true

Loading autochecks from /omd/sites/mysite/var/check_mk/autochecks/MYHOST.mk
 + "Migrating fs_used name" failed
Traceback (most recent call last):
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 1700, in close
    self.parser.Parse("", 1) # end of data
xml.parsers.expat.ExpatError: no element found: line 1, column 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/omd/sites/mysite/lib/python3/cmk/update_config.py", line 158, in run
    step_func()
  File "/omd/sites/mysite/lib/python3/cmk/update_config.py", line 199, in _update_fs_used_name
    cmk.update_rrd_fs_names.update()
  File "/omd/sites/mysite/lib/python3/cmk/update_rrd_fs_names.py", line 186, in update
    update_service_info(config_cache, get_hostnames(config_cache))
  File "/omd/sites/mysite/lib/python3/cmk/update_rrd_fs_names.py", line 176, in update_service_info
    update_files(hostname, service.description, service.item, 'pnp4nagios'))
  File "/omd/sites/mysite/lib/python3/cmk/update_rrd_fs_names.py", line 84, in update_files
    metrics = get_metrics(filepath, source)
  File "/omd/sites/mysite/lib/python3/cmk/update_rrd_fs_names.py", line 74, in get_metrics
    root = ET.parse(filepath).getroot()
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 1202, in parse
    tree.parse(source, parser)
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 602, in parse
    self._root = parser.close()
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 1702, in close
    self._raiseerror(v)
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 1602, in _raiseerror
    raise err
  File "<string>", line None
xml.etree.ElementTree.ParseError: no element found: line 1, column 0
Traceback (most recent call last):
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 1700, in close
    self.parser.Parse("", 1) # end of data
xml.parsers.expat.ExpatError: no element found: line 1, column 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/omd/sites/mysite/bin/cmk-update-config", line 12, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/omd/sites/mysite/lib/python3/cmk/update_config.py", line 1213, in main
    has_errors = UpdateConfig(logger, arguments).run()
  File "/omd/sites/mysite/lib/python3/cmk/update_config.py", line 158, in run
    step_func()
  File "/omd/sites/mysite/lib/python3/cmk/update_config.py", line 199, in _update_fs_used_name
    cmk.update_rrd_fs_names.update()
  File "/omd/sites/mysite/lib/python3/cmk/update_rrd_fs_names.py", line 186, in update
    update_service_info(config_cache, get_hostnames(config_cache))
  File "/omd/sites/mysite/lib/python3/cmk/update_rrd_fs_names.py", line 176, in update_service_info
    update_files(hostname, service.description, service.item, 'pnp4nagios'))
  File "/omd/sites/mysite/lib/python3/cmk/update_rrd_fs_names.py", line 84, in update_files
    metrics = get_metrics(filepath, source)
  File "/omd/sites/mysite/lib/python3/cmk/update_rrd_fs_names.py", line 74, in get_metrics
    root = ET.parse(filepath).getroot()
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 1202, in parse
    tree.parse(source, parser)
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 602, in parse
    self._root = parser.close()
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 1702, in close
    self._raiseerror(v)
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 1602, in _raiseerror
    raise err
xml.etree.ElementTree.ParseError: no element found: line 1, column 0

How to debug

...

Info

In the first line of the cmk-update-config -vv --debug output there is already a hint to the corresponding host:

Code Block

language	bash
theme	RDark

Loading autochecks from /omd/sites/mysite/var/check_mk/autochecks/MYHOST.mk

Please fetch that name and remove the .mk suffix. Now you have the Hostname. In my case it's MYHOST.

Now you can use xmllint to debug that:

Code Block

language	bash
theme	RDark

OMD[mysite]:~$ xmllint  -noout --format var/pnp4nagios/perfdata/MYHOST/*xml
var/pnp4nagios/perfdata/MYHOST/Filesystem_E__.xml:1: parser error : Document is empty

^

It seems that there is one xml file with no correpsonding rrd file and this xml file is also empty. Please remove that file and run cmk-update-config -vv --debug again

...

Code Block

language	bash
theme	RDark
collapse	true

 Loading autochecks from /omd/sites/mysite/var/check_mk/autochecks/MYHOST.mk
 + "Migrating fs_used name" failed
Traceback (most recent call last):
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 1700, in close
    self.parser.Parse("", 1) # end of data
xml.parsers.expat.ExpatError: unclosed token: line 114, column 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/omd/sites/mysite/lib/python3/cmk/update_config.py", line 158, in run
    step_func()
  File "/omd/sites/mysite/lib/python3/cmk/update_config.py", line 199, in _update_fs_used_name
    cmk.update_rrd_fs_names.update()
  File "/omd/sites/mysite/lib/python3/cmk/update_rrd_fs_names.py", line 186, in update
    update_service_info(config_cache, get_hostnames(config_cache))
  File "/omd/sites/mysite/lib/python3/cmk/update_rrd_fs_names.py", line 176, in update_service_info
    update_files(hostname, service.description, service.item, 'pnp4nagios'))
  File "/omd/sites/mysite/lib/python3/cmk/update_rrd_fs_names.py", line 84, in update_files
    metrics = get_metrics(filepath, source)
  File "/omd/sites/mysite/lib/python3/cmk/update_rrd_fs_names.py", line 74, in get_metrics
    root = ET.parse(filepath).getroot()
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 1202, in parse
    tree.parse(source, parser)
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 602, in parse
    self._root = parser.close()
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 1702, in close
    self._raiseerror(v)
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 1602, in _raiseerror
    raise err
  File "<string>", line None
xml.etree.ElementTree.ParseError: unclosed token: line 114, column 0
Traceback (most recent call last):
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 1700, in close
    self.parser.Parse("", 1) # end of data
xml.parsers.expat.ExpatError: unclosed token: line 114, column 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/omd/sites/mysite/bin/cmk-update-config", line 12, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/omd/sites/mysite/lib/python3/cmk/update_config.py", line 1213, in main
    has_errors = UpdateConfig(logger, arguments).run()
  File "/omd/sites/mysite/lib/python3/cmk/update_config.py", line 158, in run
    step_func()
  File "/omd/sites/mysite/lib/python3/cmk/update_config.py", line 199, in _update_fs_used_name
    cmk.update_rrd_fs_names.update()
  File "/omd/sites/mysite/lib/python3/cmk/update_rrd_fs_names.py", line 186, in update
    update_service_info(config_cache, get_hostnames(config_cache))
  File "/omd/sites/mysite/lib/python3/cmk/update_rrd_fs_names.py", line 176, in update_service_info
    update_files(hostname, service.description, service.item, 'pnp4nagios'))
  File "/omd/sites/mysite/lib/python3/cmk/update_rrd_fs_names.py", line 84, in update_files
    metrics = get_metrics(filepath, source)
  File "/omd/sites/mysite/lib/python3/cmk/update_rrd_fs_names.py", line 74, in get_metrics
    root = ET.parse(filepath).getroot()
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 1202, in parse
    tree.parse(source, parser)
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 602, in parse
    self._root = parser.close()
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 1702, in close
    self._raiseerror(v)
  File "/omd/sites/mysite/lib/python3.8/xml/etree/ElementTree.py", line 1602, in _raiseerror
    raise err
xml.etree.ElementTree.ParseError: unclosed token: line 114, column 0
OMD[mysite]:~$

How to debug

...

Info

In the first line of the cmk-update-config -vv --debug output there is already a hint to the corresponding host:

Code Block

language	bash
theme	RDark

Loading autochecks from /omd/sites/mysite/var/check_mk/autochecks/MYHOST.mk

Please fetch that name and remove the .mk suffix. Now you have the Hostname. In my case it's MYHOST.

Now you can use xmllint to debug that:

Code Block

language	bash
theme	RDark

OMD[mysite]:~$ xmllint  -noout --format var/pnp4nagios/perfdata/MYHOST/*xml
var/pnp4nagios/perfdata/MYHOST/Filesystem_F__.xml:114: parser error : expected '>'
</N
   ^

It seems that there is one xml file with no correpsonding rrd file and this xml file is also empty. Please remove that file and like the problem is in row 114 with a missing '>'. Lets check the xml file

Code Block

language	bash
theme	RDark

OMD[mysite]:sed -n '2p; 114p' var/pnp4nagios/perfdata/MYHOST/Filesystem_F__.xml 
<NAGIOS>
</N

And here we go. Row 114 seems to be uncomplete.
Please add "</NAGIOS" and save the file.

The file should now look like this:

Code Block

language	bash
theme	RDark

OMD[mysite]:~$ sed -n '2p; 114p' var/pnp4nagios/perfdata/MYHOST/Filesystem_F__.xml 
<NAGIOS>
</NAGIOS

If you now run cmk-update-config -vv --debug again, there should be no error

Versions Compared

Old Version 26

New Version 27

Key

How to debug

How to debug

How to debug

How to debug

How to debug