| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Support
Forum: Website Support
Thread: Getting Statistics for old WUs |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 11
|
|
| Author |
|
|
imakuni
Advanced Cruncher Joined: Jun 11, 2009 Post Count: 105 Status: Offline Project Badges: 
|
Assuming I have a WU's name, is there a way to actively get it's statistics, instead of praying for it to show up in the "My Results" page?
----------------------------------------Man... those sorting filters are so bad........  Want to have an image of yourself like this on? Check this thread: https://secure.worldcommunitygrid.org/forums/wcg/viewthread_thread,29840 |
||
|
|
Sgt.Joe
Ace Cruncher USA Joined: Jul 4, 2006 Post Count: 7846 Status: Offline Project Badges: 
|
Yes, there is a way to get the work unit's statistics, but only for a limited time until it leaves the WCG database. I use a query I run twice a day using EXCEL. Under the data tab, select "import external data" and then "new web query". The directions for setting up the query are at: https://secure.worldcommunitygrid.org/help/viewTopic.do?shortName=api
----------------------------------------Hope this helps. Cheers
Sgt. Joe
*Minnesota Crunchers* |
||
|
|
imakuni
Advanced Cruncher Joined: Jun 11, 2009 Post Count: 105 Status: Offline Project Badges:
|
Yes, there is a way to get the work unit's statistics, but only for a limited time until it leaves the WCG database. I use a query I run twice a day using EXCEL. Under the data tab, select "import external data" and then "new web query". The directions for setting up the query are at: https://secure.worldcommunitygrid.org/help/viewTopic.do?shortName=api Hope this helps. Cheers Yes, it already helps a lot. Still, I'm not that good with excel. Sure, I was able to retireve the data and sort it manually, but it took so long. Is there a way to configure Excel to do it all for me, so I just press a button (say, update query) and I get everything done? I'd want something like this: 1- I'm only interested in one of the devices, I don't need the others. 2- I also don't need ALL that data. Via API, I already filtetered only the results that were successful, so I only need WU Name, CPU Time, Elapsed Time, Credit Claimed, and Credit Granted. No need for all those other numbers. 3- It would also look pretty if the colunms were already sorted in that order..... oh, and getting rid of potential duplicate data. Want to have an image of yourself like this on? Check this thread: https://secure.worldcommunitygrid.org/forums/wcg/viewthread_thread,29840 |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
hmmm, I've had multiple queries defined, each for the maximum block of 250, 251-500, 501-750, 751-1000, and get 900 records with Excel, filtering out those I don't want, for all in all in under 1 minute. Sorting you can define in the query on which key field you wish. See past forum discussions for previously offered defenitions. Never a missing record, never a duplicate record as some have reported occurring, which is possible, since you're querying a live database with millions of transactions per day.
|
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Just been playing with the Result Status API again, and expanding on using the ModTime field.
ModTime: Return results which were last modified on or after this time. This value is a Unixtimestamp (number of seconds since midnight Jan 1 1970). It's brilliant, as it will fetch you any results that had a change to include PV of previous stats periods, that converted to valid or another state in the current stats period, so you can track how much the totals will be for the current 12/24 hour period [Validated today so to speak]. Take the number, e.g. 1434776206 and convert the seconds to years+fraction, add to that 1.1.1970 (datevalue of 25569 in excel), then filter on the outcome if having changed after 42175 [which is today on the WCG system]. So 1434776206 =[@ModTime]/60/60/24+25569 = 15-06-20 04:56 (using custom format) Refreshing the query, it computes now that for Linux device had 1 day and 20.81 hours added to the FAHV stats for 44 results (sumifs and countifs functions). Come the noon stats, change the filter to > 42175.504167 (June 20, 12:06 UTC) and noon validations will be extracted... and so on. With the API can now even extract which results validated in that 6 minute nightly spill period. Never visiting the RS pages again.  |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
@techs, could the ModTime field in the API be added to the SortBy option?
SortBy: Defines the sorting order of the results. Options are: DeviceId, SentTime, ReportDeadline, ReceivedTime or CpuTime. Default is SentTime. When fetching data, it would allow to put older results that were previously in a Pending state to show at the top, as when the result goes valid, the ModTime is being updated to the time the result was validated. Most useful during Badge hunt end game time. |
||
|
|
uplinger
Former World Community Grid Tech Joined: May 23, 2005 Post Count: 3952 Status: Offline Project Badges: 
|
Sek,
I have added this to our list of changes to make. Thanks, -Uplinger |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Uplinger, there's a [My] problem with the ModTime. After spending weeks and weeks on reverse engineering what pirogue's WCGDAWS tool did and much more [you killed it, yes you did], and testing it against WCG stats, discovered that for a task once Valid [or any other final result state], the ModTime is not final. It get's updated when late results get returned, when a repair goes No Reply, or still comes in and gets evaluated for outcome... all copies of a task get their ModTime updated to the latest returned result time. This led to Valids on my database being pulled again into the current stats period list, much to my dismay. A few examples of a result distribution:
OET1_ 0000982_ xEBGP-FA_ rig_ 63806_ 5-- 719 Valid 6/28/15 15:27:03 6/29/15 12:31:33 0.00 47.7 / 49.6 OET1_ 0000982_ xEBGP-FA_ rig_ 63806_ 4-- 719 Valid 6/25/15 03:26:10 6/29/15 01:54:57 10.05 31.9 / 49.6 Counted in Morning, as quorum 2 occurred. OET1_ 0000982_ xEBGP-FA_ rig_ 63806_ 3-- 719 Error 6/25/15 03:20:13 6/25/15 03:23:05 0.00 48.3 / 0.0 OET1_ 0000982_ xEBGP-FA_ rig_ 63806_ 2-- 719 Error 6/25/15 03:14:52 6/25/15 03:19:05 0.00 48.3 / 0.0 OET1_ 0000982_ xEBGP-FA_ rig_ 63806_ 1-- 719 Error 6/25/15 02:56:32 6/25/15 03:13:42 0.16 1.3 / 0.0 OET1_ 0000982_ xEBGP-FA_ rig_ 63806_ 0-- 719 Valid 6/25/15 02:55:45 6/27/15 12:17:14 10.97 67.3 / 49.6 OET1_ 0000977_ xEBGP-FA_ rig_ 64862_ 5-- - No Reply 6/26/15 17:37:06 6/30/15 05:37:06 0.00 0.0 / 0.0 OET1_ 0000977_ xEBGP-FA_ rig_ 64862_ 4-- 719 Valid 6/24/15 06:20:55 6/26/15 10:18:14 11.14 99.9 / 114.9 Counted on 26th OET1_ 0000977_ xEBGP-FA_ rig_ 64862_ 3-- 719 Valid 6/23/15 05:36:05 6/27/15 04:59:18 9.00 129.9 / 114.9 OET1_ 0000977_ xEBGP-FA_ rig_ 64862_ 2-- 719 Error 6/23/15 05:31:34 6/23/15 05:35:21 0.00 45.0 / 0.0 OET1_ 0000977_ xEBGP-FA_ rig_ 64862_ 1-- 719 Error 6/23/15 05:25:04 6/23/15 05:29:14 0.00 0.0 / 0.0 OET1_ 0000977_ xEBGP-FA_ rig_ 64862_ 0-- 719 Error 6/23/15 05:24:16 6/24/15 06:18:44 2.26 30.1 / 0.0 As the WCG stats do not add these again [duh], there must be some other field that records this on your system... 'If credit granted, flag/timestamp'. Surely this existing, could this be included in the API output? So near, yet so far away. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
It is what we are looking much thanks
|
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Have sidestepped the problem of valid tasks still seeing modtime changes by splitting the DB in 2 tables, Archive and Active [what comes with each API query. If a result with grant or final outcome is on Archive, ignore for current period calculation [+/- a semi short list of exceptions].
|
||
|
|
|