Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 89
Posts: 89   Pages: 9   [ Previous Page | 1 2 3 4 5 6 7 8 9 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 463774 times and has 88 replies Next Thread
jonnieb-uk
Ace Cruncher
England
Joined: Nov 30, 2011
Post Count: 6105
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Screen Scrapers - Please Discuss

@ knreed

Great work with the APIs to make accessible Statistics By Country.

I am particularly pleased to note that I can, for example, obtain the details for all
74,781 United States members i.e. There is no artificial limit to the data.

That being the case would it be possible to remove (or substantiallly increase) the 5,000 member cap that is currently in place on the APIs for Member Statistics as noted in my post here?
----------------------------------------

To Join follow this link: Join the UK Team All Welcome! UK Team thread
[Feb 20, 2014 10:17:52 AM]   Link   Report threatening or abusive post: please login first  Go to top 
SNURK
Veteran Cruncher
The Netherlands
Joined: Nov 26, 2007
Post Count: 1217
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Screen Scrapers - Please Discuss


That would get my thumbs up too!! smile


These API's are now available.

As is the Result Status as documented here: https://secure.worldcommunitygrid.org/help/viewTopic.do?shortName=api#665


What is causing me problems is this:
With team stats I can see how many records there are by looking up the <AllTimeMembers> tag. For instance here:
http://www.worldcommunitygrid.org/team/viewTe...=P2F0MZPZ7R1&xml=true
I see <AllTimeMembers> = 20949 (as of today)
This is exactly the same as the member count in the header here (Records 1 to 50 - 20949 total):
http://www.worldcommunitygrid.org/team/viewTe...pu&teamId=P2F0MZPZ7R1

However this is not the case for country stats. For instance here:
http://www.worldcommunitygrid.org/stat/viewCo...untryCode=NL&xml=true
I see <TotalMembers> = 5306
However here:
http://www.worldcommunitygrid.org/stat/viewCo...rt=cpu&countryCode=NL
I see a different number: Records 1 to 50 - 5262 total

When I was scraping the HTML pages I read the total from this last header, and from this total I knew when I reached the last page. But now I can't do that because I can't look up this 5262 figure in XML anywhere. If I try hard enough I think I can get around this problem somehow, but it still beats me why these two totals are not the same.

Thanks,
SNURK
----------------------------------------
[Feb 21, 2014 10:10:44 PM]   Link   Report threatening or abusive post: please login first  Go to top 
jonnieb-uk
Ace Cruncher
England
Joined: Nov 30, 2011
Post Count: 6105
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Screen Scrapers - Please Discuss

SNURK

I suspect the difference in the two figures is the DOAs i.e. Signed up but never returned any result.

pornopiraat ranked RT #5,262 returned 0:000:00:00:02 RT 0 Pts 2 Res

Stats for individual teams include DOAs Stats by Country do not.

Hope that helps. wink
----------------------------------------

To Join follow this link: Join the UK Team All Welcome! UK Team thread
[Feb 21, 2014 10:38:12 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SNURK
Veteran Cruncher
The Netherlands
Joined: Nov 26, 2007
Post Count: 1217
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Screen Scrapers - Please Discuss

I like to report that there's (I think) something missing in the team members XML stats, as compared to the country members XML stats.

In country members XML stats, sorted on runtime, I see a tag <RunTimeRank> for each member:
http://www.worldcommunitygrid.org/stat/viewCo...untryCode=NL&xml=true

On the other hand, in team members XML stat, also sorted on runtime, there's no such tag:
http://www.worldcommunitygrid.org/team/viewTe...d=6BWD4Z9132&xml=true

Would be very helpfull if this tag was added there too. biggrin

greetings,
SNURK
----------------------------------------
[Mar 15, 2014 10:47:40 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SNURK
Veteran Cruncher
The Netherlands
Joined: Nov 26, 2007
Post Count: 1217
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Screen Scrapers - Please Discuss

@ knreed
I would like to know if my previous post is going to be handled by the development team. I'm converting my signature tool to use the new XML pages, but further development is hanging on the above issue right now.

Thanks,
SNURK
----------------------------------------
[Mar 18, 2014 9:21:33 AM]   Link   Report threatening or abusive post: please login first  Go to top 
jonnieb-uk
Ace Cruncher
England
Joined: Nov 30, 2011
Post Count: 6105
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Screen Scrapers - Please Discuss

For the Development team

As per SNURK's post (#2 above)

In Statistics by Members the output files for Yesterday have a field tag e.g. <RunTimeRank> which is not present in the All Time data files.

It would also be helpful if a similar tag were used in the Statistics by Teams and in the output files for individual teams example

Thanks
----------------------------------------

To Join follow this link: Join the UK Team All Welcome! UK Team thread
[Mar 21, 2014 8:13:59 PM]   Link   Report threatening or abusive post: please login first  Go to top 
pirogue
Veteran Cruncher
USA
Joined: Dec 8, 2008
Post Count: 685
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Screen Scrapers - Please Discuss

Why doesn't the XML data match the results page data?
According to the XML, this WU was due 2 days before it was sent.

API XML:
<Result><AppName>mcm1</AppName>
<ClaimedCredit>0.0</ClaimedCredit><CpuTime>0.0</CpuTime>
<ElapsedTime>0.0</ElapsedTime><ExitStatus>0</ExitStatus>
<GrantedCredit>0.0</GrantedCredit>
<DeviceId>1664333</DeviceId>
<DeviceName>LTAD844275</DeviceName>
<ModTime>1396063877</ModTime>
<Name>MCM1_0003217_4203_2</Name>
<Outcome>0</Outcome>
<ReceivedTime>null</ReceivedTime>
<ReportDeadline>2014-03-27T21:58:59</ReportDeadline>
<SentTime>2014-03-29T03:31:17</SentTime>
<ServerState>4</ServerState>
<ValidateState>0</ValidateState>
</Result>


Result page:
MCM1_ 0003217_ 4203_ 2-- LTAD844275 In Progress 3/29/14 03:31:17 3/31/14 14:19:17 0.00 / 0.00 0.0 / 0.0
----------------------------------------

[Mar 29, 2014 3:45:41 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Screen Scrapers - Please Discuss

Could it be the work unit generation timestamp you can see in the header? If so, wrong label. Example where the unit is generated 9 dys before sending

Workunit Status

Project Name: Mapping Cancer Markers
Created: 03/19/2014 04:51:27
Name: MCM1_0003200_0584
Minimum Quorum: 2
Replication: 2


Result Name App Version Number Status Sent Time Time Due /
Return Time CPU Time / Elapsed Time (hours) Claimed/ Granted BOINC Credit
MCM1_ 0003200_ 0584_ 2-- 732 Valid 3/28/14 07:33:58 3/29/14 08:14:37 11.86 224.7 / 235.4
----------------------------------------
[Edit 1 times, last edit by Former Member at Mar 29, 2014 8:22:24 AM]
[Mar 29, 2014 8:12:49 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Screen Scrapers - Please Discuss

What am I doing wrong, as unable to replicate the pirogue's xml pull from the result status page. When following the instructions in help just get the web page as is in the browser, the format is supposed to be:

https://secure.worldcommunitygrid.org/ms/view...mp;pageNum=1&xml=true

This is the help article:

Search Results for: xml

Do you have an XML interface for the statistics?
You may obtain an XML version for most of the public statistics pages by simply adding either "?xml=true" if there are no other parameters in the URL or "&xml=true" if there already are parameters on the URL for the page.

BTW, for the web developers. When prying in help at https://secure.worldcommunitygrid.org/help/viewHelp.do the 'Search for Help' text in the search box has first to erased/deleted before the search term can be entered. If you don't you get for instance 'xmlSearch for Help' where the cursor is placed default at start of box. The 'Search for Help' appears as normal black text. Would have expected that to be grey or something and to disappear upon first keystroke.
[Mar 29, 2014 12:58:22 PM]   Link   Report threatening or abusive post: please login first  Go to top 
pirogue
Veteran Cruncher
USA
Joined: Dec 8, 2008
Post Count: 685
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Screen Scrapers - Please Discuss

What am I doing wrong, as unable to replicate the pirogue's xml pull from the result status page. When following the instructions in help just get the web page as is in the browser, the format is supposed to be:

https://secure.worldcommunitygrid.org/ms/view...mp;pageNum=1&xml=true

See here: https://secure.worldcommunitygrid.org/help/viewTopic.do?shortName=api#665


For the individual results, you need to use:
https:/ /secure.worldcommunitygrid.org/api/members/{your-member-name}/results?code={your-verification-code}&format=xml

----------------------------------------

[Mar 29, 2014 8:03:31 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 89   Pages: 9   [ Previous Page | 1 2 3 4 5 6 7 8 9 | Next Page ]
[ Jump to Last Post ]
Post new Thread