| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Support
Forum: Website Support
Thread: Can the WCG make it possible for Free-DC Stats to get "fresh stats"? |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 49
|
|
| Author |
|
|
Johnny Cool
Ace Cruncher USA Joined: Jul 28, 2005 Post Count: 8621 Status: Offline Project Badges: 
|
From the Free-DC Stats forum:
----------------------------------------It's difficult. WCG do not provide any kind of mechanism for 3rd party stats to extract data from. So, the only option is to grab the individual webpages. Problem is that this is not very reliable with WCG. Often times they are half updated or missing altogether. My stats engine basically disables all users, then as it parses each one it is enabled (a little more complicated than that but that's a good enough explanation..) in order to track name movement. So if a page is half updated, half the users are missing etc. Those users are then flagged as disabled. They then reappear later maybe a couple of days later. It all adds up to chaos... Somehow, there must be a way. We can always hope.  |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Well, that's wrong. Direct the person here, and we will help him out.
|
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges:
|
We provide the large sized data file extracts that sites like BOINC Synergy and BOINC Stats are able to use with no issue. This are available at:
http://www.worldcommunitygrid.org/boinc/stats/ We also have XML interfaces to most of the public facing stats pages. Please see: http://www.worldcommunitygrid.org/help/viewTopic.do?shortName=web#329 What are they looking for beyond this? The XML interfaces for the website pages are subject to the stats update unavailable times. They need to make sure they include checks for this. The large file extracts are not subject to the stats update unavailable times. |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
The BOINC stats addon for Firefox certainly near real time shows the latest credits all day long as does do Vista gadgets, discussed in a Chat Room thread.
----------------------------------------
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
Movieman
Veteran Cruncher Joined: Sep 9, 2006 Post Count: 1042 Status: Offline |
I just sent BOK at free-dc a link to this page.
----------------------------------------He is the one that runs the free-dc stats page. Great guy.  |
||
|
|
Bok7575
Advanced Cruncher Joined: Jul 8, 2008 Post Count: 51 Status: Offline Project Badges: 
|
Hi,
boinc stats are no problem and never have been, I've been getting and parsing them for a long time without any issues Boinc WCG stats It was always just the 'classic' stats that were an issue. For the past 4 or 5 years I've had to do screenscrapes of the html which is not particularly easy for two reasons. 1. The html does not lend itself too well to regexp manipulation 2. The timeouts which occur occasionally during updates. I have just been informed about the xml and though this would be better, it still would involve multiple fetches (1 for each team) and in a few tests I've made the retrieval time is still pretty poor. Question I have is, should I just retire the 'classic ' stats ? I mean, from what I can see there are no longer going to be any results returned via the old UD agent (is this correct? ). I'd leave them in an archived state, so they would still be available of course, but there would be no point in updating them. Bok |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Good question, good thread. Looks like all the involved parties have checked in, contact has been made.
Stand by for countdown... By the way, let me take this opportunity to thank you all for putting time and energy into sorting out the stats and making them clear and presentable. I enjoy checking Free-DC stats to see how we are all coming along. Thanks, keep up the great work. |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Hi, boinc stats are no problem and never have been, I've been getting and parsing them for a long time without any issues Boinc WCG stats It was always just the 'classic' stats that were an issue. For the past 4 or 5 years I've had to do screenscrapes of the html which is not particularly easy for two reasons. 1. The html does not lend itself too well to regexp manipulation 2. The timeouts which occur occasionally during updates. I have just been informed about the xml and though this would be better, it still would involve multiple fetches (1 for each team) and in a few tests I've made the retrieval time is still pretty poor. Question I have is, should I just retire the 'classic ' stats ? I mean, from what I can see there are no longer going to be any results returned via the old UD agent (is this correct? ). I'd leave them in an archived state, so they would still be available of course, but there would be no point in updating them. Bok Know so much that in the background work is evolving to take out what's described as the "imbalance" adjustment. My translation is that something is afoot so that soon there will be BOINC credits appearing in the My Grid stats instead plus something to acknowledge past contributions made with the retired UD agent. Don't know if those values will port back to what's seen on the client side. There's a newsletter going out this moment. That may lift a tip off the cover. ciao
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges:
|
I have just been informed about the xml and though this would be better, it still would involve multiple fetches (1 for each team) and in a few tests I've made the retrieval time is still pretty poor. We will be moving the website DB to larger hardware later this year as well as upgrading from DB2 7.2 -> DB2 9.5. Those two changes should improve the response time (when we migrate from 7.2 -> 9.5 we will actually do a export/import and will redesign the physical aspects of the db to take advantage of the new features of DB2 9.5) Question I have is, should I just retire the 'classic ' stats ? I mean, from what I can see there are no longer going to be any results returned via the old UD agent (is this correct? ). I'd leave them in an archived state, so they would still be available of course, but there would be no point in updating them. There will be no more stats coming with the UD agent. |
||
|
|
Bok7575
Advanced Cruncher Joined: Jul 8, 2008 Post Count: 51 Status: Offline Project Badges:
|
Thanks,
----------------------------------------looks like I archive those stats then.. Out of interest do you do the mysql software with a DB2 backend rather than mysql? I'd be most interested to see how it performs comparitively...I can imagine the use of materialized views being very useful. In my daily job I do a lot of work with DB2 on mainframe as well as DB2 UDB under linux. Bok [Edit 1 times, last edit by Bok7575 at Jul 8, 2008 9:28:29 PM] |
||
|
|
|