[self-solved] Issue with Poller, RRD and interface

[anandks]

Hi All:

I am facing an issue with one of my query, log files indicates the query returns the correct values, and matches the data source templates. Poller does not indicate any undefined values. But,
rrdtool updates the the file with "U" for the 5 tail end variables, see below:

rrdtool lastupdate /var/lib/cacti/rra/176/1285.rrd
5MHZ_NET CHN RPTR_DLM_STATUS AC_FAIL AC_ON SPKRVOL CTRL_FAIL CTRL_ON ACC_DIS INTCOM PA_FAIL PA_FULL PA_TEST RX1_FAIL RX1_ACTIVE RX_LOCK RX2_FAIL RX2_LOCK RX2_ACTIVE SQ TX_INHIBIT TX_LOCK WL_FAIL WL_ON RPTR_ACT_MAST RPTR_BSI_CHN RPTR_CONTROL_CHN_CH RPTR_DISPATCH_CHN RPTR_DLAN1_STATUS RPTR_DLAN2_STATUS RPTR_RX_FREQ RPTR_TX_FREQ RPTR_PIB_STATUS RPTR_MODE RPTR_CUR RPTR_RX_STATUS RPTR_SITE_STATUS RPTR_TX_STATUS RPTR_WL_STATUS STN_REF_STATUS BOOSTER_FAIL LastConnectSuccess CheckTimeStamp

1586519104: 0 1 2 0 1 0 0 1 0 0 0 0 0 0 0 1 0 3 0 11 0 1 0 1 4 4 4 4 5 5 264.3500 271.8125 2 9 0 2 7 2 U U U U U

looking at the logs, the variables passed to the rrd2tool appears to be truncated (last variable passed is RTPR_TX_STATUS, as you can see, there are 5 more after that)

2020/04/10 07:50:16 - POLLER: Poller[1] CACTI2RRD: /usr/bin/rrdtool update /usr/share/cacti/rra/176/1285.rrd --skip-past-updates --template CHN:5MHZ_NET:RPTR_DLM_STATUS:AC_FAIL:AC_ON:SPKRVOL:CTRL_FAIL:CTRL_ON:ACC_DIS:INTCOM:PA_FULL:PA_FAIL:PA_TEST:RX1_FAIL:RX1_ACTIVE:RX_LOCK:RX2_FAIL:RX2_ACTIVE:RX2_LOCK:SQ:TX_INHIBIT:TX_LOCK:WL_FAIL:WL_ON:RPTR_ACT_MAST:RPTR_BSI_CHN:RPTR_CONTROL_CHN_CH:RPTR_DISPATCH_CHN:RPTR_DLAN1_STATUS:RPTR_DLAN2_STATUS:RPTR_RX_FREQ:RPTR_TX_FREQ:RPTR_PIB_STATUS:RPTR_MODE:RPTR_CUR:RPTR_RX_STATUS:RPTR_SITE_STATUS:RPTR_TX_STATUS 1586519405:1:0:2:0:1:0:0:1:0:0:0:0:0:0:0:1:0:0:3:11:0:1:0:1:4:4:4:4:5:5:264.3500:271.8125:2:9:0:2:7:2

Here is the query itself, that returns the full list:
2020/04/10 07:50:05 - SPINE: Poller[1] PID[26023] Device[176] HT[1] DS[1285] SCRIPT: perl /usr/share/cacti/scripts/query-fpm.pl , output: CHN:1 5MHZ_NET:0 RPTR_DLM_STATUS:2 AC_FAIL:0 AC_ON:1 SPKRVOL:0 CTRL_FAIL:0 CTRL_ON:1 ACC_DIS:0 INTCOM:0 PA_FULL:0 PA_FAIL:0 PA_TEST:0 RX1_FAIL:0 RX1_ACTIVE:0 RX_LOCK:1 RX2_FAIL:0 RX2_ACTIVE:0 RX2_LOCK:3 SQ:11 TX_INHIBIT:0 TX_LOCK:1 WL_FAIL:0 WL_ON:1 RPTR_ACT_MAST:4 RPTR_BSI_CHN:4 RPTR_CONTROL_CHN_CHN:4 RPTR_DISPATCH_CHN:4 RPTR_DLAN1_STATUS:5 RPTR_DLAN2_STATUS:5 RPTR_RX_FREQ:264.3500 RPTR_TX_FREQ:271.8125 RPTR_PIB_STATUS:2 RPTR_MODE:9 RPTR_CUR:0 RPTR_RX_STATUS:2 RPTR_SITE_STATUS:7 RPTR_TX_STATUS:2 RPTR_WL_STATUS:2 STN_REF_STATUS:8 BOOSTER_FAIL:3 LastConnectSuccess:1 CheckTimeStamp:1586519283

In addition, decided to have a look at the DataSource template - and the interface is all jumbled up (See screenshot), I don't know how to work on the template, with the buttons overlayed on variable tabs. This is the case with IE, Edge, Chrome and FireFox

Cacti Version: 1.2.10 (unable to upgrade to 1.2.11 - can't get past 42% see post viewtopic.php?f=2&t=60847 )

Capture.PNG (70.88 KiB) Viewed 1031 times

[anandks]

managed to get 1.2.11 installed by pulling it from github, BUT, still the problem exists.

[anandks]

anyone having some pointers to look for on this, please ? I am still struggling to figure this out. I can report back, that I have second host that I brought up online exported/imported templates that is having issue , and the same issue - the debug indicates, when it gets from cacti2rrd, the data source string is truncated to 410 characters in length -

[anandks]

this is definitely related to string lengths. Started going through the exported templates and found data source names truncated to 19 characters. Step-by-step I was able to get this to work
by changing all the data source names to be less than 19 and it also appears that he total string length that is passed from poller also affects the outcome.

I used to have this.

2020/04/10 07:50:05 - SPINE: Poller[1] PID[26023] Device[176] HT[1] DS[1285] SCRIPT: perl /usr/share/cacti/scripts/query-fpm.pl , output: CHN:1 5MHZ_NET:0 RPTR_DLM_STATUS:2 AC_FAIL:0 AC_ON:1 SPKRVOL:0 CTRL_FAIL:0 CTRL_ON:1 ACC_DIS:0 INTCOM:0 PA_FULL:0 PA_FAIL:0 PA_TEST:0 RX1_FAIL:0 RX1_ACTIVE:0 RX_LOCK:1 RX2_FAIL:0 RX2_ACTIVE:0 RX2_LOCK:3 SQ:11 TX_INHIBIT:0 TX_LOCK:1 WL_FAIL:0 WL_ON:1 RPTR_ACT_MAST:4 RPTR_BSI_CHN:4 RPTR_CONTROL_CHN_CHN:4 RPTR_DISPATCH_CHN:4 RPTR_DLAN1_STATUS:5 RPTR_DLAN2_STATUS:5 RPTR_RX_FREQ:264.3500 RPTR_TX_FREQ:271.8125 RPTR_PIB_STATUS:2 RPTR_MODE:9 RPTR_CUR:0 RPTR_RX_STATUS:2 RPTR_SITE_STATUS:7 RPTR_TX_STATUS:2 RPTR_WL_STATUS:2 STN_REF_STATUS:8 BOOSTER_FAIL:3 LastConnectSuccess:1 CheckTimeStamp:1586519283

by reducing the datasource names, and removing "_" I had for clarity, ended up with

2020/04/15 12:45:06 - SPINE: Poller[1] PID[32023] Device[176] HT[1] DS[1898] SCRIPT: perl /usr/share/cacti/scripts/query-fpm.pl , output: CHN:1 5MHZNET:0 RPTRDLMSTS:2 ACFAIL:0 ACON:1 SPKRVOL:0 CTRLFAIL:0 CTRLON:1 ACCDIS:0 INTCOM:0 PAFULL:0 PAFAIL:0 PATEST:0 RX1FAIL:0 RX1ACTIVE:0 RXLOCK:1 RX2FAIL:0 RX2ACTIVE:0 RX2LOCK:3 SQ:11 TXINHIBIT:0 TXLOCK:1 WLFAIL:0 WLON:1 RPTRACTMAST:4 RPTRBSICHN:4 RPTRCTRLCHN:4 RPTRDSPCHCHN:4 RPTRDLAN1STS:5 RPTRDLAN2STS:5 RPTRRXFREQ:264.3500 RPTRTXFREQ:271.8125 RPTRPIBSTS:2 RPTRMODE:9 RPTRCUR:0 RPTRRXSTS:2 RPTRSITESTS:7 RPTRTXSTS:2 RPTRWLSTS:2 STNREFSTS:8 BOOSTERFAIL:3 LastSuccess:1 ChkTmStmp:1586968983


and this works., now I realize pretty much many of queries that had long names are all int he same condition.

Update - It appears, that if the total string length of the variables is above 386, any variables after that are truncated.

[TheWitness]

It's good that there are some people who self solve. After all Open Source means mostly that. I personally deal with some really large installs, and we try to avoid anything with such wide column widths. This isn't as much of a problem as in earlier years due to changes in both Cacti and MySQL/MariaDB design, but it still looks ugly from the UI. The normal solution is something like:

ALTER TABLE poller_output MODIFY COLUMN output VARCHAR(2048) NOT NULL default "";
ALTER TABLE poller_output_boost MODIFY COLUMN output VARCHAR(2048) NOT NULL default "";

Now that Cacti is using InnoDB with a dynamic row format, things like this won't be as much of a problem.

[anandks]

It's good that there are some people who self solve. After all Open Source means mostly that. I personally deal with some really large installs, and we try to avoid anything with such wide column widths. This isn't as much of a problem as in earlier years due to changes in both Cacti and MySQL/MariaDB design, but it still looks ugly from the UI. The normal solution is something like:

ALTER TABLE poller_output MODIFY COLUMN output VARCHAR(2048) NOT NULL default "";
ALTER TABLE poller_output_boost MODIFY COLUMN output VARCHAR(2048) NOT NULL default "";

Now that Cacti is using InnoDB with a dynamic row format, things like this won't be as much of a problem.

Thank you, when I never had the problem of the same query in 0.8.8 , now I see that poller_output was a TEXT data type (65,535 characters) vs now it is VARCHAR (512)..

I understand what you mean by large variable names, there were many reasons other than just readability, the devices being monitored were legacy equipment with serial data stream. segmenting them into multiple graphs would mean, multiple poll events, which I wanted to avoid, etc. It would be nice, if there is a CLI script that can import a delimited file with parameters to create data source templates.

[TheWitness]

There is GitHub for that one, though it might get a low priority without some assistance via a pull request.