Some Weathermaps not updating

[saguaro]

I have been running Cacti and weathermap for some time. I recently installed cacti on a newer server. It was running fine until one day I saw a few maps with old create dates on them. 99% are still updating fine. All the maps that show stale create dates actually have updated rrd time stamps and show current rrd graph info. From what I can tell I saw another poller running with the same timestamp of the day that some of the graphs stopped updating. If I check the output directory the html, png, thumb.png files for the graph have the same old timestamp of when the second poller process was found.

Will a rebuild poller cache delete all historical rrd graph data?
Is there a way to regen a single map to fix this issue?
I have tried php weathermap --config mapname.conf but this returned a blank map output when clicked on thumbnail image.

Any info appreciated


Plugin Management Cacti Version: 0.8.7g, Plugin Architecture Version: 2.8
WeatherMap 0.97a

Red Hat Enterprise Linux Server release 6.7 (Santiago)
PHP 5.3.3 (cli) (built: Oct 23 2014 06:58:46)
Server version: Apache/2.2.15 (Unix)

[Howie]

Rule #1 of weathermap (and any other) troubleshooting: What do the logs say?

Weathermap doesn't look at the poller cache at all, so that won't be an issue. The data you see in the popup graphs is being drawn by Cacti not weathermap, so that's an indication that the cacti poller is running OK (which I guess you knew - your graphs are working in Cacti). So...

* is it the maps towards the end of the list that aren't updating? (i.e. is something killing the map generation process, perhaps?)

* Is it the same few maps each time?

To generate all the maps, the same as the poller would, you can run "php weathermap-cacti-rebuild.php", but be careful of the permissions in the output directory so that the poller can still write to those files. If any errors are generated, you should see them on the console.

But first - check the logs. Most issues generate a log line. The main one that doesn't is hitting the memory_limit, where PHP will just kill the process. However, there is one main point in the process where this is likely, and weathermap logs something like "if this is the last line in your logs...". If that *is* the last (weathermap) line in your logs, you may want to increase memory_limit in php.ini

[saguaro]

Yes of course the logs, A cat of the log reveals it stopped writing to those files of the related map on 3/16 . Log is updating as normal. The map is not near the bottom of the map list and other maps below it are updating the png and html files with the current create date of every 5 min poll cycle.

cat cacti.log | grep ORG
03/16/2017 08:52:40 AM - WEATHERMAP: Poller[0] [Map 227] ORGNNY.conf: Map: /var/www/html/cacti/plugins/weathermap/configs/ORGNNY.conf -> /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.html & /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.png
03/16/2017 08:52:40 AM - WEATHERMAP: Poller[0] [Map 227] ORGNNY.conf: About to write image file. If this is the last message in your log, increase memory_limit in php.ini [WMPOLL01]
03/16/2017 08:52:41 AM - WEATHERMAP: Poller[0] [Map 227] ORGNNY.conf: Wrote map to /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.png and /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.thumb.png

cat cacti.log | grep ERRORS
cat cacti.log | grep errors

Oddly enough I have the original old system still running and it too has many more stale maps on the same date.

Permissions and cron are all run by root on both systems and have been for sometime without issues. I will run the php weathermap-cacti-rebuild.php on the old system to see if corrects. TY

[Howie]

Just so I understand for sure: are you saying that /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.png is not being updated even though weathermap thinks it is?

(also as far as the logs are concerned, I think errors will show with WARN, not ERRORS - I would grep for WEATHERMAP though)

[saguaro]

I ran weathermap from the cli on the old system with less cpu.
I know there are many maps that need to be looked into with WARNINGS, But I didnt think that weathermap was taking this long to complete with WARNINGS. I didnt grep for WEATHERMAPbecause like I said there are many that need attention.

The poller takes under 3 minutes on both systems so I thought weathermap was happy all the same. I am focusing on this one because it was updating until the 16th on the new system.

When I watched weathermap run I disabled polling and cleared the logs so I could just get the weathermap log data. I saw that it took 5 minutes on some maps.
I was going to disable the maps that were taking a long time but as you can see this went on much longer than expected.

Yes correct, ERRORS are with the POLLER only not with WEATHERMAP.
Yes it seams to be the same maps that stopped updating.


This is from the old system, it completes , On the new system I cant find anything that says run complete. But again most maps are updating.

OLD SYSTEM WEATHERMAP START
03/20/2017 03:32:23 PM - WEATHERMAP: Poller[0] Weathermap 0.97a starting - Normal logging mode. Turn on DEBUG in Cacti for more information
03/20/2017 11:42:25 PM - WEATHERMAP: Poller[0] STATS: Weathermap 0.97a run complete - Mon, 20 Mar 17 23:42:25 -0400: 193 maps were run in 29402 seconds with 6746 warnings.
END


OLD SYSTEM - weathermap cli run it does update the map
cat cacti.log | grep ORG | more

03/20/2017 08:28:34 PM - WEATHERMAP: Poller[0] [Map 227] ORGNNY.conf: Map: /opt/lampp/htdocs/cacti/plugins/weathermap/configs/ORGNNY.conf -> /opt/lampp/htdocs/cacti/
plugins/weathermap/output/386cffef2965c1f1825d.html & /opt/lampp/htdocs/cacti/plugins/weathermap/output/386cffef2965c1f1825d.png
03/20/2017 08:28:34 PM - WEATHERMAP: Poller[0] WARNING: [Map 227] ORGNNY.conf: ReadData: LINK node03090_copy-node03090_4, target: /opt/lampp/htdocs/cacti/rra/rtr101_
wan_hcvlny_cv_net_traffic_in_12297.rrd on config line 96 of /opt/lampp/htdocs/cacti/plugins/weathermap/configs/ORGNNY.conf had no valid data, according to WeatherMap
DataSource_rrd
03/20/2017 08:28:34 PM - WEATHERMAP: Poller[0] WARNING: [Map 227] ORGNNY.conf: ReadData: LINK node03090_copy-node03090_copy_copy_copy, target: /opt/lampp/htdocs/cact
i/rra/rtr101_wan_hcvlny_cv_net_traffic_in_13121.rrd on config line 104 of /opt/lampp/htdocs/cacti/plugins/weathermap/configs/ORGNNY.conf had no valid data, according
to WeatherMapDataSource_rrd
03/20/2017 08:28:34 PM - WEATHERMAP: Poller[0] WARNING: [Map 227] ORGNNY.conf: ReadData: LINK node03090_copy-node03090_copy_copy, target: /opt/lampp/htdocs/cacti/rra
/rtr101_wan_hcvlny_cv_net_traffic_in_14111.rrd on config line 112 of /opt/lampp/htdocs/cacti/plugins/weathermap/configs/ORGNNY.conf had no valid data, according to W
eatherMapDataSource_rrd
03/20/2017 08:28:34 PM - WEATHERMAP: Poller[0] [Map 227] ORGNNY.conf: About to write image file. If this is the last message in your log, increase memory_limit in ph
p.ini [WMPOLL01]
03/20/2017 08:28:34 PM - WEATHERMAP: Poller[0] [Map 227] ORGNNY.conf: Wrote map to /opt/lampp/htdocs/cacti/plugins/weathermap/output/386cffef2965c1f1825d.png and /op
t/lampp/htdocs/cacti/plugins/weathermap/output/386cffef2965c1f1825d.thumb.png


NEW SYSTEM
From the logs on the new system , I dont see warnings for this map. It only says it wrote map but the time stamps say otherwise.

ls -l | grep 386cffef2965c1f1825d
-rw-r--r-- 1 root root 22268 Mar 16 08:52 386cffef2965c1f1825d.html
-rw-r--r-- 1 root root 43330 Mar 16 08:52 386cffef2965c1f1825d.png
-rw-r--r-- 1 root root 34923 Mar 16 08:52 386cffef2965c1f1825d.thumb.png

03/16/2017 08:37:27 AM - WEATHERMAP: Poller[0] [Map 227] ORGNNY.conf: Wrote map to /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.png and /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.thumb.png
03/16/2017 08:42:24 AM - WEATHERMAP: Poller[0] [Map 227] ORGNNY.conf: Map: /var/www/html/cacti/plugins/weathermap/configs/ORGNNY.conf -> /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.html & /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.png
03/16/2017 08:42:25 AM - WEATHERMAP: Poller[0] [Map 227] ORGNNY.conf: About to write image file. If this is the last message in your log, increase memory_limit in php.ini [WMPOLL01]
03/16/2017 08:42:25 AM - WEATHERMAP: Poller[0] [Map 227] ORGNNY.conf: Wrote map to /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.png and /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.thumb.png
03/16/2017 08:47:23 AM - WEATHERMAP: Poller[0] [Map 227] ORGNNY.conf: Map: /var/www/html/cacti/plugins/weathermap/configs/ORGNNY.conf -> /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.html & /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.png
03/16/2017 08:47:24 AM - WEATHERMAP: Poller[0] [Map 227] ORGNNY.conf: About to write image file. If this is the last message in your log, increase memory_limit in php.ini [WMPOLL01]
03/16/2017 08:47:24 AM - WEATHERMAP: Poller[0] [Map 227] ORGNNY.conf: Wrote map to /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.png and /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.thumb.png
03/16/2017 08:52:40 AM - WEATHERMAP: Poller[0] [Map 227] ORGNNY.conf: Map: /var/www/html/cacti/plugins/weathermap/configs/ORGNNY.conf -> /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.html & /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.png
03/16/2017 08:52:40 AM - WEATHERMAP: Poller[0] [Map 227] ORGNNY.conf: About to write image file. If this is the last message in your log, increase memory_limit in php.ini [WMPOLL01]
03/16/2017 08:52:41 AM - WEATHERMAP: Poller[0] [Map 227] ORGNNY.conf: Wrote map to /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.png and /var/www/html/cacti/plugins/weathermap/output/386cffef2965c1f1825d.thumb.png

[Howie]

Code: Select all

03/20/2017 11:42:25 PM - WEATHERMAP: Poller[0] STATS: Weathermap 0.97a run complete - Mon, 20 Mar 17 23:42:25 -0400: 193 maps were run in 29402 seconds with 6746 warnings.

Wow, this is crazy!

In the manage maps page, it shows the warnings per map. Are those all (or mostly) in one map?

Would you be able to share your largest map? (by PM if you prefer, I don't share user's data)

In the newer version, it logs the time per map. In 0.97a it calculates that value but doesn't store it, maybe a small patch will help figure out where the time is going.

[saguaro]

Howie - Lets just say we are putting all your hard work through the motions
Unfortunately cleanup with these maps were not happening. They are built by a few people. I don't build the maps but have been tasked with getting things back on track. I thought all was good on the new system until 3/16. For the most part like I said most maps are still updating fine. Sorry no cant share any data too many hostnames.

No the errors are not with one map the Average is a 1-5 errors on some maps..... 201 maps total . the point was to get it on a newer system and then start the cleanup....

So I guess you will like this STAT then.
03/21/2017 10:11:51 AM - SYSTEM STATS: Time:109.5891 Method:cmd.php Processes:16 Threads:N/A Hosts:939 HostsPerProcess:59 DataSources:22713 RRDsProcessed:11063

[saguaro]

Actually now that I look the old system has many more warning per map than it did in the past, don't know why. It had less than the new system. The new system has 197 warnings.

On the old system, I am disabling the maps with high warnings and rerunning weathermap. As I see it running it is still hanging for a few minutes on some graphs.
That is where the time is going.

[saguaro]

Even tho the poller time is under 5 minutes, I show it over running the 5 minutes. I know you said weathermap doesn't rely on the poller,
should I just split this system into 2 systems and be done with it?

New System
ps -ef | grep poller
root 1643 5977 0 12:32 pts/0 00:00:00 grep poller
root 25850 25848 0 12:25 ? 00:00:00 /bin/sh -c /usr/bin/php /var/www/html/cacti/poller.php > /dev/null 2>&1
root 25851 25850 64 12:25 ? 00:04:57 /usr/bin/php /var/www/html/cacti/poller.php
root 30723 30721 0 12:30 ? 00:00:00 /bin/sh -c /usr/bin/php /var/www/html/cacti/poller.php > /dev/null 2>&1
root 30726 30723 32 12:30 ? 00:00:52 /usr/bin/php /var/www/html/cacti/poller.php
ps -ef | grep poller
root 1676 5977 0 12:32 pts/0 00:00:00 grep poller
root 30723 30721 0 12:30 ? 00:00:00 /bin/sh -c /usr/bin/php /var/www/html/cacti/poller.php > /dev/null 2>&1
root 30726 30723 33 12:30 ? 00:00:55 /usr/bin/php /var/www/html/cacti/poller.php

03/21/2017 12:31:50 PM - SYSTEM STATS: Time:109.5334 Method:cmd.php Processes:16 Threads:N/A Hosts:939 HostsPerProcess:59 DataSources:22713 RRDsProcessed:11094
03/21/2017 12:31:53 PM - SYSTEM DSSTATS STATS: Type:HOURLY, Time:2.0250

[Howie]

No, I said it doesn't rely on the poller cache. It runs from the poller. It doesn't usually read data via the poller (unless you've turned that on specifically with rrd_use_poller_output aka "boost support" and are using rrd targets). I see that you are using dsstats - weathermap can also read dsstats data directly from the database, which should be faster than running lots of external rrdtool processes. There is a tool in the random-bits folder that might help you convert a map or two, just to see if it helps.

The time in the final SYSTEM STATS line for the poller should include whatever time Weathermap took to run. You should also be seeing a WEATHERMAP STATS line with a shorter time. As I mentioned before, the weathermap poller function does actually calculate a duration for each map, but it doesn't store that anywhere. I'll make sure it does for the next release. It *does* log that number though, when DEBUG logging is turned on. Unfortunately, that will probably slow down the system a bit. With DEBUG on, you'll see lines from this:

debug("TIME: $mapfile took $map_duration seconds.\n");

in the cacti.log

So weathermap needs to be able to either read the files, or access the data some other way. In a Cacti 1.0 system, I don't actually know how that stuff works, but I'd assume that the master server is the one that has to run Weathermap (and you'll need the next release of Weathermap for it to work at all).

[saguaro]

Ok, I see the boost plugin is shown but not installed. We are using dsstats to create MAX graphs, will keep that in mind as another source for weathermaps.

Tried a few things on the old system, Disabled all maps but 2 to see time completion. Its seems to consistently wait for the MAX config maps to complete adding to the overall completion time. Even with 33 errors on a non max graph it takes only 1 second. When I run weathermap and include the MAX graph, I get a 2 minute delay for completion time of the max graph. 2 minutes added to over 100 maps is a long time for completion. I will turn on debugging next to see more info. Log example below;

03/24/2017 09:41:54 AM - WEATHERMAP: Poller[0] Weathermap 0.97a starting - Normal logging mode. Turn on DEBUG in Cacti for more information

03/24/2017 09:41:54 AM - WEATHERMAP: Poller[0] [Map 197] Core.conf: Map: /opt/lampp/htdocs/cacti/plugins/weathermap/configs/Core.conf -> /opt/lampp/htdocs/cacti/plugins/weathermap/output/b639dcb2be408b7c6d68.html & /opt/lampp/htdocs/cacti/plugins/weathermap/output/b639dcb2be408b7c6d68.png
03/24/2017 09:41:54 AM - WEATHERMAP: Poller[0] WARNING: [Map 197] Core.conf: ReadData: LINK node05721.net, target: /opt/lampp/htdocs/cacti/rra/nodeXX_traffic_in_6876.rrd on config line 190 of /opt/lampp/htdocs/cacti/plugins/weathermap/configs/Core.conf had no valid data, according to WeatherMapDataSource_rrd
------33 errors laaaater------
03/24/2017 09:41:54 AM - WEATHERMAP: Poller[0] [Map 197] Core.conf: About to write image file. If this is the last message in your log, increase memory_limit in php.ini [WMPOLL01]
03/24/2017 09:41:55 AM - WEATHERMAP: Poller[0] [Map 197] Core.conf: Wrote map to /opt/lampp/htdocs/cacti/plugins/weathermap/output/b639dcb2be408b7c6d68.png and /opt/lampp/htdocs/cacti/plugins/weathermap/output/b639dcb2be408b7c6d68.thumb.png
03/24/2017 09:41:55 AM - WEATHERMAP: Poller[0] STATS: Weathermap 0.97a run complete - Fri, 24 Mar 17 09:41:55 -0400: 1 maps were run in 1 seconds with 33 warnings.

--- Now with MAX graph included -----

03/24/2017 09:52:01 AM - WEATHERMAP: Poller[0] Weathermap 0.97a starting - Normal logging mode. Turn on DEBUG in Cacti for more information

03/24/2017 09:52:01 AM - WEATHERMAP: Poller[0] [Map 197] Core.conf: Map: /opt/lampp/htdocs/cacti/plugins/weathermap/configs/Core.conf -> /opt/lampp/htdocs/cacti/plugins/weathermap/output/b639dcb2be408b7c6d68.html & /opt/lampp/htdocs/cacti/plugins/weathermap/output/b639dcb2be408b7c6d68.png
------33 errors laaaater------
03/24/2017 09:52:02 AM - WEATHERMAP: Poller[0] [Map 197] Core.conf: Wrote map to /opt/lampp/htdocs/cacti/plugins/weathermap/output/b639dcb2be408b7c6d68.png and /opt/lampp/htdocs/cacti/plugins/weathermap/output/b639dcb2be408b7c6d68.thumb.png
03/24/2017 09:52:02 AM - WEATHERMAP: Poller[0] [Map 200] CoreMAX.conf: Map: /opt/lampp/htdocs/cacti/plugins/weathermap/configs/CoreMAX.conf -> /opt/lampp/htdocs/cacti/plugins/weathermap/output/6f8640167ba382aa06ad.html & /opt/lampp/htdocs/cacti/plugins/weathermap/output/6f8640167ba382aa06ad.png

03/24/2017 09:54:37 AM - WEATHERMAP: Poller[0] [Map 200] CoreMAX.conf: About to write image file. If this is the last message in your log, increase memory_limit in php.ini [WMPOLL01]
03/24/2017 09:54:37 AM - WEATHERMAP: Poller[0] [Map 200] CoreMAX.conf: Wrote map to /opt/lampp/htdocs/cacti/plugins/weathermap/output/6f8640167ba382aa06ad.png and /opt/lampp/htdocs/cacti/plugins/weathermap/output/6f8640167ba382aa06ad.thumb.png
03/24/2017 09:54:37 AM - WEATHERMAP: Poller[0] STATS: Weathermap 0.97a run complete - Fri, 24 Mar 17 09:54:37 -0400: 2 maps were run in 156 seconds with 33 warnings.

This raises a couple of questions;
What would be causing the MAX graphs to delay the weathermap completion times?
Is there a quick way to resolve warnings with WeatherMapDataSource_rrd like below? I have over 100 maps to look into.....
Is there an easy button to validate data sources without having to edit every map config file?

03/24/2017 09:41:54 AM - WEATHERMAP: Poller[0] WARNING: [Map 197] Core.conf: ReadData: LINK node05721.net, target: /opt/lampp/htdocs/cacti/rra/nodeXX_traffic_in_6876.rrd on config line 190 of /opt/lampp/htdocs/cacti/plugins/weathermap/configs/Core.conf had no valid data, according to WeatherMapDataSource_rrd

Does and upgrade path require to patch the system 1 patch at a time or are patches/upgrades cumulative in that I can just use the last patch?
As a coder myself, I understand the answer is not always an easy one but I will throw it out there anyway, is there an approx ETA for weathermap for Cacti 1.0?
TY

[Howie]

Can you clarify what you mean by "max graphs"? Weathermap doesn't draw graphs. If you mean using dsstats targets, can you post the config for one please? They each perform one database query - if anything they should be faster than regular rrdtool targets. That's why I mentioned convert-dsstats.php before.

For the warnings - it's not telling you that the target is incorrect. There's no "validation" to be done in that sense. It's saying that it asked rrdtool to get data from the last few minutes (I think it's 15 by default), and didn't get any data, but NaNs. If you look at that file, has it been updated in the last while? If you look at the graph generated from the data in cacti, does that have a line for the most recent times?

So...

Code: Select all

ls -l  /opt/lampp/htdocs/cacti/rra/nodeXX_traffic_in_6876.rrd
rrdtool info  /opt/lampp/htdocs/cacti/rra/nodeXX_traffic_in_6876.rrd

should tell you when it last received useful data.


(You can get a time breakdown per-map without DEBUG logging, by changing line 220 of lib/poller-common.php to begin warn instead of debug, by the way, in 0.97a)

[saguaro]

So we have a concept of maps that make a map of MAX graph data only. I guess these are DSSTAT Targets. A script basically takes a regular map.conf file and creates a copy and substitutes the TARGET rrds with dsstats. I edited to protect the innocent hostnames. These are a line from the standard conf file and then max.conf conf file I have been testing with. They don't reflect the rrd from which I had no valid data from the previous post. I will list that data at the bottom of this post.

CORE.conf
LINK some host name info here - I Edited
WIDTH 4
INFOURL /cacti/graph.php?rra_id=all&local_graph_id=14860&rra_id=all
OVERLIBGRAPH /cacti/graph_image.php?local_graph_id=14860&rra_id=0
TARGET /opt/lampp/htdocs/cacti/rra/I Edited hostnames here_traffic_in_15289.rrd
NODES I Edited hostnames here
VIA 1005 770
BANDWIDTH 100000M


COREMAX.conf
LINK some host name info here - I Edited
WIDTH 4
INFOURL /cacti/graph.php?rra_id=all&local_graph_id=14860&rra_id=all
OVERLIBGRAPH /cacti/graph_image.php?local_graph_id=14860&rra_id=0
TARGET 8*dsstats:dailypeaktraffic_in:traffic_out
NODES I Edited hostnames here
VIA 1005 770
BANDWIDTH 100000M



I edited the time in the poller as you suggested. Its shows the same map time as the poller stats runtime.

03/24/2017 03:10:51 PM - WEATHERMAP: Poller[0] Weathermap 0.97a starting - Normal logging mode. Turn on DEBUG in Cacti for more information
03/24/2017 03:10:51 PM - WEATHERMAP: Poller[0] [Map 199] CoreMAX.conf: Map: /opt/lampp/htdocs/cacti/plugins/weathermap/configs/CoreMAX.conf -> /opt/lampp/htdocs/cacti/plugins/weathermap/output/e9219fa4a95ab7f3cf80.html & /opt/lampp/htdocs/cacti/plugins/weathermap/output/e9219fa4a95ab7f3cf80.png
03/24/2017 03:14:25 PM - WEATHERMAP: Poller[0] [Map 199] CoreMAX.conf: About to write image file. If this is the last message in your log, increase memory_limit in php.ini [WMPOLL01]
03/24/2017 03:14:25 PM - WEATHERMAP: Poller[0] [Map 199] CoreMAX.conf: Wrote map to /opt/lampp/htdocs/cacti/plugins/weathermap/output/e9219fa4a95ab7f3cf80.png and /opt/lampp/htdocs/cacti/plugins/weathermap/output/e9219fa4a95ab7f3cf80.thumb.png
03/24/2017 03:14:25 PM - WEATHERMAP: Poller[0] WARNING: [Map 199] CoreMAX.conf: TIME: /opt/lampp/htdocs/cacti/plugins/weathermap/configs/CoreMAX.conf took 214 seconds.
03/24/2017 03:14:25 PM - WEATHERMAP: Poller[0] STATS: Weathermap 0.97a run complete - Fri, 24 Mar 17 15:14:25 -0400: 1 maps were run in 214 seconds
with 1 warnings.


RRD DATA reported with no valid data from previous post - different map - Time stamp is from when I ran the test this morning;
-bash-3.2# ls -l /opt/lampp/htdocs/cacti/rra/nodeXX_net_traffic_in_6876.rrd
-rw-r--r-- 1 root root 94816 Mar 24 11:15 /opt/lampp/htdocs/cacti/rra/nodeXX_net_traffic_in_6876.rrd


XX_Core.conf: TARGET /opt/lampp/htdocs/cacti/rra/nodeXX_traffic_in_6876.rrd
-bash-3.2# rrdtool info /opt/lampp/htdocs/cacti/rra/nodeXX_traffic_in_6876.rrd

filename = "/opt/lampp/htdocs/cacti/rra/nodeXX_traffic_in_6876.rrd"
rrd_version = "0003"
step = 300
last_update = 1490368501
ds[traffic_in].type = "COUNTER"
ds[traffic_in].minimal_heartbeat = 600
ds[traffic_in].min = 0.0000000000e+00
ds[traffic_in].max = 2.0000000000e+10
ds[traffic_in].last_ds = "5160135470132300"
ds[traffic_in].value = NaN
ds[traffic_in].unknown_sec = 1
ds[traffic_out].type = "COUNTER"
ds[traffic_out].minimal_heartbeat = 600
ds[traffic_out].min = 0.0000000000e+00
ds[traffic_out].max = 2.0000000000e+10
ds[traffic_out].last_ds = "48946098079978851"
ds[traffic_out].value = NaN
ds[traffic_out].unknown_sec = 1
rra[0].cf = "AVERAGE"
rra[0].rows = 600
rra[0].cur_row = 216
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[0].cdp_prep[1].value = NaN
rra[0].cdp_prep[1].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 700
rra[1].cur_row = 674
rra[1].pdp_per_row = 6
rra[1].xff = 5.0000000000e-01
rra[1].cdp_prep[0].value = NaN
rra[1].cdp_prep[0].unknown_datapoints = 3
rra[1].cdp_prep[1].value = NaN
rra[1].cdp_prep[1].unknown_datapoints = 3
rra[2].cf = "AVERAGE"
rra[2].rows = 775
rra[2].cur_row = 569
rra[2].pdp_per_row = 24
rra[2].xff = 5.0000000000e-01
rra[2].cdp_prep[0].value = NaN
rra[2].cdp_prep[0].unknown_datapoints = 15
rra[2].cdp_prep[1].value = NaN
rra[2].cdp_prep[1].unknown_datapoints = 15
rra[3].cf = "AVERAGE"
rra[3].rows = 797
rra[3].cur_row = 43
rra[3].pdp_per_row = 288
rra[3].xff = 5.0000000000e-01
rra[3].cdp_prep[0].value = 1.7938606891e+10
rra[3].cdp_prep[0].unknown_datapoints = 28
rra[3].cdp_prep[1].value = 2.3076930249e+11
rra[3].cdp_prep[1].unknown_datapoints = 28
rra[4].cf = "MAX"
rra[4].rows = 600
rra[4].cur_row = 272
rra[4].pdp_per_row = 1
rra[4].xff = 5.0000000000e-01
rra[4].cdp_prep[0].value = NaN
rra[4].cdp_prep[0].unknown_datapoints = 0
rra[4].cdp_prep[1].value = NaN
rra[4].cdp_prep[1].unknown_datapoints = 0
rra[5].cf = "MAX"
rra[5].rows = 700
rra[5].cur_row = 664
rra[5].pdp_per_row = 6
rra[5].xff = 5.0000000000e-01
rra[5].cdp_prep[0].value = NaN
rra[5].cdp_prep[0].unknown_datapoints = 3
rra[5].cdp_prep[1].value = NaN
rra[5].cdp_prep[1].unknown_datapoints = 3
rra[6].cf = "MAX"
rra[6].rows = 775
rra[6].cur_row = 33
rra[6].pdp_per_row = 24
rra[6].xff = 5.0000000000e-01
rra[6].cdp_prep[0].value = NaN
rra[6].cdp_prep[0].unknown_datapoints = 15
rra[6].cdp_prep[1].value = NaN
rra[6].cdp_prep[1].unknown_datapoints = 15
rra[7].cf = "MAX"
rra[7].rows = 797
rra[7].cur_row = 421
rra[7].pdp_per_row = 288
rra[7].xff = 5.0000000000e-01
rra[7].cdp_prep[0].value = 1.9765329200e+08
rra[7].cdp_prep[0].unknown_datapoints = 28
rra[7].cdp_prep[1].value = 3.1975755278e+09
rra[7].cdp_prep[1].unknown_datapoints = 28

[Howie]

ds[traffic_in].value = NaN

ds[traffic_out].value = NaN

This is what I meant. This is what "had no valid data" is telling you.

Assuming you are in EDT timezone, the file date and the last_update time match up, but at what time did you check this?

[Howie]

To get further with the performance issue, you would need to look at DEBUG level logging for those maps. They definitely shouldn't take that long.

One simple test would be to run the same map from the command line with no data collection, just to be sure it's not something else (I doubt it):

Code: Select all

  time ./weathermap --no-data --config configs/your-file-name.conf --output temp.png

I would expect this to finish in a second or less, unless your map is very funky.