Some data sources (RRD files) aren't created by poller

[stefanbrudny]

Screenshot_20220630_234633.png (173.56 KiB) Viewed 1972 times

Code: Select all

mysql> select pi.rrd_step, pi.rrd_next_step, data_source_profile_id, count(*) 
    -> from poller_item as pi 
    -> inner join data_template_data AS dtd 
    -> on dtd.local_data_id=pi.local_data_id 
    -> group by pi.rrd_step, pi.rrd_next_step, data_source_profile_id;
+----------+---------------+------------------------+----------+
| rrd_step | rrd_next_step | data_source_profile_id | count(*) |
+----------+---------------+------------------------+----------+
|      300 |           120 |                      9 |      320 |
|      300 |           180 |                      9 |      318 |
|      300 |           240 |                      9 |      316 |
|      300 |             0 |                      9 |      320 |
|      300 |            60 |                      9 |      318 |
|       60 |             0 |                      3 |       30 |
|       60 |             0 |                     10 |        2 |
+----------+---------------+------------------------+----------+
7 rows in set (0.01 sec)

[TheWitness]

Okay, the way I read this is as follow:

1) Data Source Profile 9 has a poller interval of 5 minutes
2) Data Source Profiles 3 & 10 have a poller interval of 1 minute

When you understand how the poller runs, basically, every poller pass, the rrd_next_step will be reduced by poller_interval or 60 seconds in your case. When the rrd_next_step goes to 0 or negative, the Data Source will be polled.

So, for all your Data Sources with a poller interval of 60 seconds and a cron interval of 300, to spread the load, they are distributed over 5 minutes. So, after 5 minutes of polling, your boost cache should have data for all of the Data Sources. So, you have to give it time to run for 10 or 20 minutes, and then check to see that all the local_data_id's are populated.

Setup boost so that it only runs once an hour or so to give you time to diagnose. A few ways to watch this is to watch a specific local_data_id and verify that the rrd_next_step is decrementing every poller interval.

[stefanbrudny]

Thank you very much,

honestly I do understand partially, so far mostly by examining outgoing SNMP queries in tcpdump but I promise to try go deeper...

Anyhow, I will set up boost to 60 minutes (had 30 minutes) and check the results / continue tomorrow. I can also resign from 1 and 5 minute polling, but... this configuration is very cool and, simply, why should I limit expectations.

[TheWitness]

Appreciate you investing time. Already fixed a few bugs along the way.

[stefanbrudny]

I haven't got any luck to fix it. Problem persists and its nagging. I could swear I've seen it last time (5 years ago, but no prove). But I like to move things forward.

* tried using my "clean" LXC template, but results are same. Apparently not so clean and it would indicate the error is at the initial configuration.
* so I wanted to prove there is a bug in stock (devel version), installed from scratch and the result is on the screen. Spoiler alret: failed, works.

Lefft hand: firefox with Cacti, data sources missing. Right hand: chrome, everything is correct. If in doubts, browser is of no importance (feature request: give a name for Cacti so I can distinguish between the two, such as PROD, TEST, STG (staging)).

So I belive there is a misconfiguration or a bug related to some unexpected configuration. Right hand is really very fresh, not even holding any data. Really not sure why after smooth clean install first task is tto check why poller first message is 2022-07-02 01:20:00 - POLLER: Poller[Main Poller] PID[17053] Maximum runtime of 298 seconds exceeded. Exiting.

I'll try to build on it and make snapshots to determine which configuration breaks it. However, it might be more difficult than I belive it is as there is a lot to import from the 1st cacti to 2nd cacti.

Screenshot_20220702_011124.png (631.59 KiB) Viewed 1948 times

[TheWitness]

Upload the LXC Template you were discussing.

[stefanbrudny]

Sent in PM.

[stefanbrudny]

hmmm aparently PM stuck in Outbox in the forum engine, but this is positive, as I finally got some time and made a little progress.

It appears that I am using some wrong order in creation of the templates or sth else yet, as I recreated by diverging and testing at each step from default templates. That approach results again in no working poller (timeouting), graph not rendered, but after tinkering with delete-recreate-same trial I finally got it working. I don't know exactly what is wrong, but I am close and will use more time for that (success like a drug).

[TheWitness]

I'm back from vacation, could do a webex or something if you are up for it. Best on the weekends.

[stefanbrudny]

...and I am back from covid. Sending hours in pm.

[TheWitness]

Wife and I got it in January. No VAXX, no problem. We had our Horse Paste, Quercetin, Zinc, Vitamin D3, and K2. We also nebulized highly diluted food grade HO. NSAIDS we're important, and Zpack just in case.

Came out of it fine, though the cough took a while to clear. Glad you are better.

[TheWitness]

I'm on a business trip starting tomorrow. Let me know.