| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Support
Forum: Website Support
Thread: API for devices list |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 15
|
|
| Author |
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
"First all I wanted was to scrape the data from My Contribution in two ways:"
Likely you know, but this has an API since a few years accessible via membername/verification code... no 'scraping' needed. The pull allows splitting the project stats, the totals/rankings, the average and the base member info- |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Hi lavaflow,
While it's true that you can get the "My Contribution" page data via the API, I've not found that it's possible to get the historic data pages, namely the "My Contribution History" or the "Device Statistics History" page data via API. Those only seem possible via screen scrape. If you've discovered a method to get these two via the API, that would be great! I'd really prefer not to have to resort to screen scraping if at all possible.  Best, -mark |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
3 have been on my API wish-list for longer, MC history Statistics, Device Profile Manager device listing and Device Statistics. Those require scrape, which suffer frequent time out, and thus looped with a back-off delay not to trigger WCG's bot protection.
|
||
|
|
adriverhoef
Master Cruncher The Netherlands Joined: Apr 3, 2009 Post Count: 2346 Status: Recently Active Project Badges: 
|
"First all I wanted was to scrape the data from My Contribution in two ways:" Likely you know, but this has an API since a few years accessible via membername/verification code... no 'scraping' needed. The pull allows splitting the project stats, the totals/rankings, the average and the base member info- It's amazing how this failed to attract my attention over the years. I could also note - when I tried this API call at 00:34:00 UTC, while the My Contribution page just said "The statistics update will finish in about 100 minutes 31 seconds ." - that the API call was already able to tell me my new contribution statistics (e.g. that I moved one place ahead in RunTime - from #1383 to #1382 - as well as in Points - from #400 to #399). What also could be noticed is that this API call doesn't list the team totals (see "My Team"), it just lists your own contribution to the teams in which you ever participated. Nevertheless, this is nice to know, so I've added the option -S to wcgresults to retrieve these statistics through the API in XML format.  As a team captain, to retrieve your team's totals, you still need to scrape the website to retrieve these statistics. [Edit 2 times, last edit by adriverhoef at May 25, 2019 1:04:44 AM] |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Good for you and at that you get 'your' team history to boot.
----------------------------------------The API data call is never locked, bar for maintenance. Theoretically you can test on a timer if an A selected value has changed, and then process the rest of your scripts the moment the change has happened and then auto-publish. BTW the team data is exported every (24 hours?) and only for the BOINC credits, not the legacy points. It's a big xml. BOINCStats says the Team data was last pulled 2019-05-24 23:38:01 UTC (08:40:12 old) [Edit 3 times, last edit by Former Member at May 25, 2019 10:51:19 AM] |
||
|
|
|