netsaint_statd as means of collecting data from remote syste

[integr8er]

Hi,

Am using Cacti for LOTs of stuff. In a bank where netsaint is used to monitor systems and send alerts/pages for problems. I found it is really easy to add functions to the netsaint_statd client daemon. It is just a perl script that runs as a daemon. You send it a command and it does the data collection and returns info as a single line of text ready to parse. Actually, I use a helper script on the cacti side to do the talking to netsaint_statd. It just opens the socket and sends the command and netsaint_statd replies with the data. This allows me to use the program for alert monitoring and cacti for trending/graphing. If anyone wants to know more, I'll include the script. I'm counting processes, adding up their sizes (VSZ in the ps command), adding up the accumulated CPU time for selected processes and counting records (lines) in log files, getting vmstat info, sysload from the uptime command and I even added an external command function so I do not have to extend the netsaint_statd code each time I have a need. This works really well - for me anyway...

[guest]

Hi,

I'd be very interested in this. I used to use netsaint in my former job and I've not had the time yet to set it up again. Please mail your code to jhenkel at sterlingcrane dot ca
Thanks

[ciordia9]

As an avid netsaintien please post or link us to your helpers, tips, or tricks. Love to add more consolidation to graphing. woo!

-C9-

[frederic]

I am also very interested,

can you please send me your scripts ?

Thanks in advance.

fmedico at yahoo dot com

[Guest]

Give me a little time and I'll post this stuff somewhere. It would be a tar file with the netsaint_statd perl script and a couple of very small utility command scripts included. I want to cleanup the code and comments a little more to keep biz names and such out of it. Rather than email to requesting users... Maybe I can put it up on a web site for easy download. Watch this message board entry for updates on this... Don;t want to set to much of an expectation, but I'll try to do this sometime this month .

FYI - I have the following functions added to the script for cacti to query (above and beyond the standard netsaint_statd provided functions.

Available Standard netsaint_statd functions.
Some not usable for Cacti.

users
<count>

disk [filesystem device]
<device> <blocks> <usage%> <mount point> -- Semi Useful

alldisks
<all disks info> -- Not Useful for cacti

uptime
<xxday(s), hh:mm - load2> -- Not very useful for cacti

procs
<count> -- Minimal - proc_count more flexible

named_proc
<count> -- Minimal value - proc_count, proc_stats_[p|b] more flexible

Available USB extended netsaint_statd functions.

close_wait
<count> -- Minimal value - netstat_conns more useful

sysload
<load1> <load2> <load3>

proc_count [process_name]
<count>

filewc filespec
<lines> <bytes>

proc_stats_b [process_name]
<pr_ttl> <sz_min> <sz_max> <sz_avg> <sz_ttl> <ct_min> <ct_max> <ct_avg> <ct_ttl>
Returns info for selected process by basename sz=size from VSZ in ps listing and ct=elapsed cpu time

proc_stats_p [perl_pattern]
<pr_ttl> <sz_min> <sz_max> <sz_avg> <sz_ttl> <ct_min> <ct_max> <ct_avg> <ct_ttl>
Returns info for selected process by matching a pattern in the args portion of the returned ps command.

vmstat12
<last line of vmstat 1 2 output with spaces compressed and leading space removed>

external_cmd [basename_of_executable [optional single argument to pass along]]
<returns last line of external command output with spaces compressed and leading space removed>
The basename_of_executable MUST be present and executable in /usr/local/netsaint_extcmd. Lets you execute crafted/unique commands on a machine without requiring the code to be written into the netsaint_statd script. You could be a bit security concious about this one. But since the script to execute MUST be placed into the target directory (and cannot be a symlink) by someone (hopefully the sysadmin) this ought to be ok.

netstat_conns
<estab> <twait> <cwait> <listen> <ostate> <tsize>
Note: You may have to add the netsaint user to various groups to allow netstat to execute. On sun, you have to add netsaint to group sys.

Nothing for IO at this point. The damn iostat command returns broadly differing data formats from one platform to another (I have to deal with at least 3). But I'm thinking I'll just total up all the io's on all disks and just return that consolidated data and plot it as overall disk bandwidth used... Or something like this... Any Ideas welcome

Also, there is a function in the program that allows you to send a command to it to cause it to update itself from a known source (the netsaint server listed in the allowed_hosts array for those who know about this program). This is awesome if you have netsaint_statd's deployed on lots of servers and they might be managed by many people. If you had to chase them (sys admins) down for each update to each machine, they would not be happy with you pretty quickly... With this function, you can have your netsaint_server be secure (hopefully eh) and provide an updated source to some remote client. You have to manually issue a commandline on the netsaint server to initiate the process, but it's a heck of a lot easier than manually installing/updating 100+ boxes the old fashioned way . Oh, the update function is fairly simple - not high tech foolproof, but does work pretty well.

And - since you might be doing a number of data collection commands that each require a ps listing command (so as to glean the info), the program will cache the ps listing for a short time of 45 seconds. Thus, it generates 1 ps listing that is used by potentially many hits. Since the data collection occurs at 5 minute intervals, this works well for efficiency and also promotes accuracy between multiple queries.

Overall, I was thinking that this should be submitted back to the netsaint distribution... I'd like to... We'll see...

[zonkimwald]

hi, i am very interestet in your script for adding basic netsaint reporting features into cacti,
and it would help me VERY MUCH if u would share these scripts to me......
yours sincerely
oliver kupke

[integr8er]

This morning, I have emailed to a few that asked for, and to Ian/Rax a small tarball (26k) containing my extensions to the netsaint_statd daemon program. Hopefully he will make this tar file available for download soon.

Since my last posting, I've added functions to collect some iostat information, a filesize function that returns a given files size (in bytes) without using wc (it uses perl -s filetest operator), so it's small and quick. This way you can watch a log files growth rates without counting the lines. I also added a remote ping function that lets you request the remote netsaint_statd client to ping some other address/hostname and return the roundtrip time min/avg/max.

I've included in the tarball some other example iostat data collecting scripts that are more comprehensive than the netsaint_statd contained iostat function. The daemon has a generalized function more suitable for a variety of platforms, while the included working sample scripts glean just about everything out of the iostat command that you can for your platforms iostat command.

Hopefully next week, I'll update these files to add in the code to work under Linux (redhat anyway). Whats here is the provided netsaint platform support, with my extensions that apply to AIX, SunOS and HP-UX. Sorry, but no thought was given to Windows.

I'd be happy to hear about experiences from anyone that uses the netsaint_statd stuff.

Thanks