| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
| World Community Grid Forums
|
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 16
|
|
| Author |
|
|
adriverhoef
Master Cruncher The Netherlands Joined: Apr 3, 2009 Post Count: 2346 Status: Offline Project Badges:
|
Maybe if I had many many machines to look after it would be a different story but like adriverhoef (mentioned above) who took weeks to reprogram his scripts I would adapt. Well, here is a word from me (i.e. adriverhoef). It should be noted that it took absolutely no time at all to reprogram my script 'wcgresults', since the parts of the API that are being used by that script (i.e. downloading the outcomes of validation, mainly) didn't change! The parts of the website that changed for me were the scraping of HTML pages that I did with my script 'wcgstats' and that cost me indeed two weeks: first off it cost me some time to find out how to login, because that has changed (I couldn't find the URL!), then base the script on the new API (defining some new programcode for one's own devices, repairing how to update the script, integrating the new task statuses, and much, much, much more), I also changed some options and redesigned the whole thing, next I needed to check all programcode and see to it that it worked OK, before it was time to (re)publish the script and make it available for downloading. I don't want to publish half work, you know. However, since wcgstats didn't work in the beginning after the new API was put into place I also wanted/needed a way to report/log my total contribution (runtime, points, results), so I added some code to wcgresults (available with the option -SSSS, maybe you've seen it being mentioned somewhere ) so that it mimicked the output of wcgstats (called without options).So, in the end, the duration that I mentioned, two weeks, was actually used for three scripts that needed rewriting: not only for wcgstats and for wcgresults - mentioned above - it also applied to wcgformat, because the field 'AVN' (Application Version Number) was removed, the time format had changed and I had to rewrite the way of detecting the difference between Results Status and Workunit Status pages. Also, when a task hadn't been returned yet, the format that was displayed was different. Took me some headscratching, too. One other script, wcgmembercmp, didn't need any rewriting, as far as I can tell. Well, one small thing, something to do with capitalization and HTML scraping. The last script. wcglog, really didn't need any rewriting at all. In any case, if there were small tweaks, they had nothing to do with the WCG website. For the curious, you can find the scripts at this place. ![]() [Edit 3 times, last edit by adriverhoef at Nov 9, 2021 11:06:49 AM] |
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges:
|
Excel, script only pulls the returned results. The script for in progress works fine. Edit: for good order, this morning I pulled a 'stable' backup from my cloud of October 13, ran the script and same problem... 'type mismatch', another result of course and long as I don't fetch a new set repeating the process always fails at same record. Note it does raw data pull, but the problem record's data then cannot be read into the database (table) as it baulks at a record that does show as text rather than numbers for the numeric fields, left aligned, as give away instead of right aligned.. I need more information than "type mismatch". The website is returning good well-formed XML that validates and I cannot see what would be causing the issue on your end. You will need to have your tool provide more information so that it can report exactly what the issue is (what data point is it failing on) and the look to see why it can't handle that data point. Without this information there is nothing I can do as I cannot see anything wrong. |
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges:
|
Regarding the [old] API... I have been using the old API to fetch results as JSON (except for the My Contribution equivalent, which only offers XML!) ... Never having fetched results as XML I did an experiment and discovered that if one follows the documentation verbatim it returns JSON when one asks for XML! If, however, one puts format=XML (lower-case F) instead of Format=XML it returns XML! (The API web page implies one should use upper-case F!) Thank you for pointing this out. I've fixed the documentation so that the first letter of the parameters is lower case as is expected by the api. |
||
|
|
alanb1951
Veteran Cruncher Joined: Jan 20, 2006 Post Count: 1317 Status: Offline Project Badges:
|
Regarding the [old] API... I have been using the old API to fetch results as JSON (except for the My Contribution equivalent, which only offers XML!) ... Never having fetched results as XML I did an experiment and discovered that if one follows the documentation verbatim it returns JSON when one asks for XML! If, however, one puts format=XML (lower-case F) instead of Format=XML it returns XML! (The API web page implies one should use upper-case F!) Thank you for pointing this out. I've fixed the documentation so that the first letter of the parameters is lower case as is expected by the api. Kevin, Thanks for that! Glad to help if/when I can :-) I do, however, find myself wondering whether that has affected anyone [else] but they've just not posted about it! -- I'm presuming lavaflow had actually looked at what was being returned by the API so wasn't bitten by this, but as a message earlier in this thread was about leaving I guess we'll never know... Cheers - Al. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Regarding the [old] API... I have been using the old API to fetch results as JSON (except for the My Contribution equivalent, which only offers XML!) ... Never having fetched results as XML I did an experiment and discovered that if one follows the documentation verbatim it returns JSON when one asks for XML! If, however, one puts format=XML (lower-case F) instead of Format=XML it returns XML! (The API web page implies one should use upper-case F!) I have no idea whether the XML format itself has changed -- what I got looked o.k. to me, with the actual results following three lines about available data (just like the JSON data!), and I've no idea how one would load it into Excel so I've no idea what might have broken for lavaflow... Of course, I'm presuming the fetched data was XML, not JSON :-) Cheers - Al. P.S. I now have scripts that use the new API as well, so if/when they turn the old API off I'm "good to go"... JIC, to follow up on F v f, the in progress and completed fetch script use the same line of code injecting parameter values depending on what I wanted to fetch. Tthe relevant part is in the line is "&format=xml&limit=1&offset="" i.e format in all small, no extraneous json data arrival. The strangeness was that the in progress routine continued to work until I shut down. Today I loaded the file as my tablet is still computing 1 at a time and now In progress fetches give the 'type mismatch' (error 13) too. same as completed, the first record is malformed, the following are all fine. Help says "Cause: The variable or property isn't of the correct type. For example, a variable that requires an integer value can't accept a string value unless the whole string can be recognized as an integer." As noted before, a non-numeric value arrives at the beginning of the first record. That's fine until that first record gets read into the table. Anyway, I wish everyone good luck in their persevering. Auf Wiederhören PS MIP will have known very long before the crunching community what was going to happen with WCG. A forced eye opener I'm sure as to revisiting the operating cost of a distributed computing effort versus a private cluster and keeping their raw data results under the hood. Krembil must have figured out the operating cost too, (did BM offer a sweetener to make this happen?). Something has to pay for this and keep it running and turn this into a win for them. Meanwhile my electricity kwh cost quadrupled. I have no sponsor, no win here. |
||
|
|
Sgt.Joe
Ace Cruncher USA Joined: Jul 4, 2006 Post Count: 7846 Status: Offline Project Badges:
|
Krembil must have figured out the operating cost too, (did BM offer a sweetener to make this happen?). Something has to pay for this and keep it running and turn this into a win for them. Meanwhile my electricity kwh cost quadrupled. I have no sponsor, no win here. You bring up an interesting point. I am going to guess Krembil as a research entity survives on the largess of not only their sponsoring institution, but also on their supporting endowments, foundation grants, government grants and other assorted donations. If they can stretch those funds by getting a significant amount of extremely cost effective computing cycles they will be willing to invest in the back end structure to keep it running. The cost/benefit ratio must be more favorable for this setup rather than having to pay for this amount of computing on their own dime. Just like you, no sponsor here. If my electric and heat bill climb dramatically this winter, I will be cutting back, but intend to keep at least one system in play (God willing). Cheers
Sgt. Joe
*Minnesota Crunchers* |
||
|
|
|