| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Support
Forum: Website Support
Thread: Can't find history |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 22
|
|
| Author |
|
|
BobbyB
Veteran Cruncher Canada Joined: Apr 25, 2020 Post Count: 638 Status: Offline Project Badges: 
|
You will find that wget eventually redirects to https://www.worldcommunitygrid.org/login/ with that wget request: same as if you tried to login without javascript turned on.
----------------------------------------Now my question is how was your script able to get just your specific stats from a wget if that is what you used. The downloadable file in "My activity" use to do just that as you can see in my post: Yes all stats combined, on overview "My Activity", instead of 1 line for each project is better (for me anyway). Part of the fun was keeping my own stats. Maybe the downloadable file in "My activity over time" (with the graphics for each project) should do that. In any case the code is there to do just that and has been changed to combine all into one line. I just deployed an update that restores this. See https://www.worldcommunitygrid.org/contribution/history [Edit 2 times, last edit by BobbyB at Sep 24, 2021 5:03:34 PM] |
||
|
|
thunder7
Senior Cruncher Netherlands Joined: Mar 6, 2013 Post Count: 238 Status: Offline Project Badges: 
|
Well, I basically did
wget $ARG --quiet --save-cookies /tmp/boinc_cookies.txt --keep-session-cookies --post-data 'j_username=<email address>&j_password=<password> https://www.worldcommunitygrid.org/j_security_check -O /tmp/boinc_login.html wget $ARG --quiet --load-cookies /tmp/boinc_cookies.txt https://secure.worldcommunitygrid.org/ms/viewMyMemberPage.do -O $MONTHDIR/$DAYPREFIX\_contribution.html wget $ARG --quiet --load-cookies /tmp/boinc_cookies.txt https://secure.worldcommunitygrid.org/ms/device/viewStatisticsByDevice.do -O $MONTHDIR/$DAYPREFIX\_device_statistics.html and then parsed the results from the contribution and device statistics pages. What URL can I use to get the statistics per project? Maybe the downloadable file in "My activity over time" (with the graphics for each project) should do that. Well, it's downloadable if I use my mouse and click on it. My problem is in getting the right url to download it automatically. |
||
|
|
BobbyB
Veteran Cruncher Canada Joined: Apr 25, 2020 Post Count: 638 Status: Offline Project Badges:
|
I see what you mean and I had never delved that much into wget so the --post-data stuff is one on me. (time to read the docs again)
----------------------------------------I tried the API and plenty of stuff comes in. https://www.worldcommunitygrid.org/help/topic.s?shortName=api Bottom line: https://www.worldcommunitygrid.org/api/members/{member name}/results?code={verification code} It might take some time for you to tune the extra options to make it output what you need and I guess it depend on how often you do it. Then happy parsing I use to take a screenshot every day and save but there is nothing to save anymore on the overview page. Then once every week or so download my stats and update my spreadsheet. [Edit 1 times, last edit by BobbyB at Sep 24, 2021 9:32:08 PM] |
||
|
|
thunder7
Senior Cruncher Netherlands Joined: Mar 6, 2013 Post Count: 238 Status: Offline Project Badges:
|
OK, that actually gets me somewhere! Thanks.
|
||
|
|
adriverhoef
Master Cruncher The Netherlands Joined: Apr 3, 2009 Post Count: 2346 Status: Offline Project Badges:
|
I tried the API and plenty of stuff comes in. https://www.worldcommunitygrid.org/help/topic.s?shortName=api Bottom line: https://www.worldcommunitygrid.org/api/members/{member name}/results?code={verification code}It might take some time for you to tune the extra options to make it output what you need Well, that is a good start and I guess we're lucky that that particular code still works, but you can't download all the data from a workunit using that code. Because with that same code you only see your own tasks, you will not find your wingmen's tasks. Let me give an example: HST1_307450_000056_AT0002_T325_F00001_S00071 Project name: Help Stop TB Created: Sep. 24, 2021 - 21:02 UTC Name: HST1_307450_000056_AT0002_T325_F00001_S00071 Minimum Quorum: 2 Replication: 2 Result name OS type Status Sent time Due / Return time CPUtime/Elapsed Claimed/Granted(edited for brevity) Here you see two tasks in a workunit. I was using a script that tried to locate a certain workunit for me and then print out its data (the Workunit Status: Project Name, Workunit Creation time, Workunit Name, Minimum Quorum, Replication, and then the info from all tasks in that workunit: Result Name, OS type, OS version, App Version Number, Status, Sent Time, Time Due/Return Time, CPU Time/Elapsed Time, Claimed/Granted BOINC Credit), optionally followed by each Result Log (output created by running the task), simply by asking to specify the Device, the name of the science (app) and the status of the task, all interactively. Worked like a charm and actually, I didn't need to browse the website to find the data. The script did it all for me. It was very handy, for it also generated all formatting for me to post the output on the forum and it would keep the transient information alive on the forum. However, with this new website design I don't know how it works  , so I'm asking for help on this matter. How do you login with a script or how do you retrieve workunit information using a script now?  [Edit 1 times, last edit by adriverhoef at Sep 25, 2021 10:37:54 PM] |
||
|
|
virtual911
Cruncher Joined: Jan 29, 2007 Post Count: 2 Status: Offline Project Badges:
|
I used to enjoy knowing my rank among donors. I don't find that info on the current website.
|
||
|
|
BobbyB
Veteran Cruncher Canada Joined: Apr 25, 2020 Post Count: 638 Status: Offline Project Badges:
|
|
||
|
|
alanb1951
Veteran Cruncher Joined: Jan 20, 2006 Post Count: 1317 Status: Offline Project Badges: 
|
Adri,
----------------------------------------However, with this new website design I don't know how it works , so I'm asking for help on this matter. How do you login with a script or how do you retrieve workunit information using a script now? You can't do it by grabbing the HTTP pages, as they use scripted routines to fetch the data from the API then format it... However, they are using the new standard API to get the data, so... I presume you've found the "Swagger" API demonstration page accessed via https://www.worldcommunitygrid.org/api/docs/index.html; the test stuff there works fine as one is already logged in, of course, and you can suss out the URLs from the examples :-) I've been playing with the new API because I already have scripts to fetch my results data using the old API and needed to make sure I could still do so with the new API. I had some success with that (though there are some interesting oddities...) I've just thrown together a Python script to see if I can get the data you seem to want: this is just "proof of concept" so the output is not all-inclusive and hasn't been formatted nicely. Here is some partial output , which I'll follow with some notes on what was needed to do this... Workunit reply information for Africa Rainfall Project (The "Mine" annotation is provided by looking at the value (true or false) of the "owner" field, by the way) To connect, one needs to do what the Login form does, which entails doing a POST to https://www.worldcommunitygrid.org/api/v3/login with your email address and password as form values for j_username and j_password respectively. Provided that works, you're good to go (at present, anyway - Kevin Reed's fix for unsolicited requests to log in simplified what I needed to do, so it could get more complex again at some point!) If you already worked out how to use the API but were held up by getting "logged in", you probably don't need the rest of this -- in that case, my apologies... For the various things needed for the report, I used the /api/v3/ms/results and /api/c3/ms/workunit parts of the API. To find out a device ID, I used /api/v3/ms/devices and to get a project ID I used /api/v3/ms/results/projects (though I did cheat and use the Swagger API page to get those!) Be aware that whilst the Swagger page formats the JSON to make it human-readable, direct use of the API gets the data as one long line. Hope the above helps... Cheers - Al. [Edit 1 times, last edit by alanb1951 at Sep 26, 2021 9:56:57 PM] |
||
|
|
adriverhoef
Master Cruncher The Netherlands Joined: Apr 3, 2009 Post Count: 2346 Status: Offline Project Badges:
|
Hi Al,
Indeed I found the Swagger API and got lost somewhere in OAuth2 authentication a.o. Fact is, I just didn't know the correct URL for authentication ever since the change to the newly designed website. Thanks to you I managed to take my first step on the new website fetching my first workunit through a scripted login. The result is indeed one long line (JSON), which of course shouldn't be that much of a problem for me.  Al, I am really, really very glad that you have taken the time to reply, especially to give the correct URL.  (The "Mine" annotation is provided by looking at the value (true or false) of the "owner" field, by the way) (Me looking at the JSON data…) Ah yes, I see! That's good to know.If you already worked out how to use the API but were held up by getting "logged in", you probably don't need the rest of this -- in that case, my apologies... Not at all, Al, no need to excuse, this has been all very helpful. Let's say we are heading in the right direction again. There's lots of work to do.Thank you very much for the great help! Adri |
||
|
|
BobbyB
Veteran Cruncher Canada Joined: Apr 25, 2020 Post Count: 638 Status: Offline Project Badges:
|
OK, so exactly where/how did you find the the "Swagger" API demonstration page". No link that I can find
|
||
|
|
|