ReportIt - 70k data items...?

[brylant]

Hi,

is there any way to add data items from command line (or any other way) - I need to add 70k interfaces...
I know this is crazy but I need to try it anyway - I need to have a report generated with interface utilization and I can't see any other way to do it (other than ReportIt)...

Thanks,

[browniebraun]

Oops,

I've seen reports with 20k data items, but 70k ..... Wow! that's a challenge!

Ok, I never tried that before, but you have to change your configuration a little bit to get that probably working with ReportIt 0.7.1:

Go to Console->Settings->Reports and increase "Maximum Execution Time (in seconds)" up to 9999 seconds. After that increase "Maximum Record Count Change" up to 100000.
Create a new report config, setup the data item presets, enable "Scheduled Reporting", enable "Auto Generated Data Items" and at least add a minimum of one data item to avoid a bug.

If you want to archive your reports automatically you have to check the configuration of your mysql server in that case: Increase variable "max_allowed_packet" up to ~16M.

Now run you report via cron or manually using the CLI and show us the results.

Best regards
-Andreas-

[brylant]

Thanks browniebraun.
I'll try it early next week (don't want to introduce untested changes before the weekend) and we'll see what's going to happen

[brylant]

Here's the update:
1st - it's "only" 35k data sources (I've got 70k interfaces and I'm collecting traffic and errors so it's actually 35k items for the report that is only using traffic - I think... .

Regarding the time:

Code: Select all

02/03/2010 10:37:18 AM - PLUGIN: Poller[0] REPORTIT STATS: Frequency:daily Time:224.7 Reports:1  Emails:0  Exports:0

I was really (I mean really) surprised as the report is really quick and that's fantastic! Thanks browniebraun.

Here's a few hints for next version:
- allow user to include/exclude NaNs from the report
- allow user to include/exclude 0s from the report (eg. do not consider interface whenever maxValue = 0)

Again - great plugin. Thanks.