| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Support
Forum: Website Support
Thread: @Techs: Now things are really [French insert] whack with the Result Status API |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 21
|
|
| Author |
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Just solved [tested with intervals of 30 minutes, and each time the Now() field showed when it last ran and new totals showed in app.
Private Sub Workbook_Open() Application.OnTime TimeValue("02:06:10"), "RefreshXML1" Application.OnTime TimeValue("14:06:10"), "RefreshXML1" End Sub Now to find a way to adapt to DST by Oct.31, something this forum software is not capable to do for it's members... stay aligned with the local system time. |
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges: 
|
We've updated the site and the FileDeleteState field is now available.
|
||
|
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
Thx for that, but as written earlier, had coded around this problem of timestamp changing, albeit not watertight. There continue to appear modtime changes even without the FileDeleteState field changing to 2 [ready to delete]. Have included the field in my app... maybe in future a little led light comes up to make it of use [simplifying code in a next version ;p]
There's another problem I've written about somewhere in passing: ServerState values missing. Refreshed the 'InProgress' pull several times, but now have even more than before 'no value' tasks listed: OET1_0001459_xSDGP-FA_rig_7866_0 25-10-2015 0:52 15-10-2015 0:52 4 FAH2_avx101122_000008_0021_003_1 18-10-2015 23:10 14-10-2015 23:10 4 FAH2_avx101121_000026_0049_003_1 18-10-2015 23:10 14-10-2015 23:10 4 1444853086 FAH2_avx38785-ls_000059_0014_003_0 18-10-2015 20:04 14-10-2015 20:04 1444851394 FAH2_avx38753-ls_000079_0007_004_0 18-10-2015 19:36 14-10-2015 19:36 1444985319 FAH2_avx38753-ls_000045_0009_004_0 18-10-2015 19:34 14-10-2015 19:34 T 1444984947 FAH2_avx38743-ls_000060_0018_004_0 18-10-2015 19:18 14-10-2015 19:18 T 1444848747 OET1_0001458_xSDGP-FA_rig_44690_2 18-10-2015 6:52 14-10-2015 18:52 1444846447 OET1_0001455_xSDGP-FA_rig_9650_2 18-10-2015 6:14 14-10-2015 18:14 1444831142 OET1_0001422_xMBGP-FA_rig_30125_2 18-10-2015 1:59 14-10-2015 13:59 1444804760 E234175_875_S.288.C39H25N3O1.AJZZRASQCUPYAV-UHFFFAOYSA-N.11_s1_14_0 24-10-2015 6:39 14-10-2015 6:39 Comparing with other listed IP tasks, can't seem to find a pattern/corellation, different sciences, different platforms, these don't have either a ServerState 4 or 5. I'm using 5 to limit the query to just returned results, and 4 to only get assignments [which include those with trickle]. Some of the above FAHB have trickles, others don't. Nothing to decipher why no ServerState=4. It's not a problem to me... I've expanded the IP query to include blank [not even a zero in the field, in the way it arrives], but it has a potential impact for the server scheduler, if it matters to you and the scheduler flow [extra load due additional passes e.g.?]. |
||
|
|
KerSamson
Master Cruncher Switzerland Joined: Jan 29, 2007 Post Count: 1684 Status: Offline Project Badges: 
|
Hi SekeRob,
----------------------------------------I have four students using the API for their Master pre-project. I would appreciate if you could contact me directly by e-mail (for e-mail address, see www.kereon.ch). In advance, I thank you. Cheers, Yves |
||
|
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
KerSamson, sorry did not see msg before: You may PM me through the Berkeley forums PM system [SekeRob]. 'Hunting Tool' being test-cased against a big crunchers user account The member volunteered to trial and report issues. So far, so good, [well, the scraping method for history pages absent xml coding is of the previous millennium]. Now scaled to handle 5,000 In Progress and 4,000 Results per cycle, which is about 99.97% of all active volunteers.
----------------------------------------While writing a lament about the scraping and researching a [no help from WCG] solution, found an ironclad way to link results status entries to devices on the device statistics, to include the uniformly named androids. This only fails if *any* new device is installed, which wont appear until it has results and then the next statistics update. But hey, can live with that, so this turned into a 'never mind, I'll hack may way around the website [highly consistent] bugs'.  TTYL [Edit 1 times, last edit by SekeRob* at Nov 2, 2015 12:37:48 PM] |
||
|
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
There is an additional field called file_delete_state that is used to control when a files associated with the result can be deleted from the filesystem. It goes from 0 -> 1 (ready for delete) -> 2 (deleted). It is not currently represented in the API but would affect the field ModTime. I'll add it to the output so that you can track it as the source of change going forward. @Kevin, returning to this field and the ever mission to cut down on records fetched, is it save to say that if a record has FileDeleteState = 1, the result has been done done for like 24 hours, at the very least not counting for the current statistics period? If so, would you consider adding this to the set of parameters of To fetch/Not To fetch i.e. if setting &FileDeleteState=0 in the url call, only those tasks are retrieved. Basically I'm now fetching the modtime through the oldest PV (as determined by previous run), but that pretty much still fetches 90+ percent of the live results, to consider that after a raw data fetch, 65-70% is deleted as not needed because they've already migrated as final to my local archive. E.g the initial ResultsAvailable value said there were 2369 from oldest PV to newest returned result. After removing all the excess to need, I'm left with 968, kind of like right after period end, with as of this moment 908 valid in the rolling past 24 hours with which I'm fine, but getting 1500 straight to rounded file is not helping. Extrapolate that to an account I'm testing against with 302K results fetched and 201K not needed. That's a difference between running for over an hour, or 15-20 minutes... the more data processed to slower it gets. Thx |
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges:
|
FileDeleteState = 1 is set based on the configured value of "delete_delay_hours" as defined here: https://boinc.berkeley.edu/trac/wiki/ProjectOptions
We usually use 24 hours but that can change at any point depending on our need for storage on the server. I would not trust it as a stable value. Also it will go from FileDeleteState=1 to 2 in a very short period of time (for example I just checked and there were only 50 results in that state). |
||
|
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
OK, so if a fileter could be set on receiving only FileDeleteState=0 That would give me only stuff that's less than 24 hours in validation state. I'm fine with that since I follow the same stats cycle as you do... every 12 hours, archive the valids, and on next pass if found to be already in archive round-file the active record entry. Mind you, that's my cycle, but others could be keeping longer spread frequencies, yet others I've read run it on an hourly schedule.
|
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges:
|
FileDeleteState can now be set as an attribute. 0, 1 and 2 are valid values for it. The FAQ has been updated with this information here: https://www.worldcommunitygrid.org/help/viewTopic.do?shortName=api#665
|
||
|
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
Excellent... will experiment with this and check out if the No-ReceivedTime when value is null works. Adding &FileDeleteState=0 to the URL will hopefully not jack the string length over 255 characters... Excel likes chopping, or throw error when this value is reached in a number of places.
---------------------------------------- edit: The FileDeleteState=0 did the job admirably. Did successive tests without and with... 1655 results before, 925 (7 passes v 4 passes at 250 per call) after adding the control and the exact same valids and pending were reported. Considering that post-fetch processing just 296 are retained, those removed including the 535 valid from previous period not yet marked FDS=0, which had moved to my local archive, a very major reduction. Now off to test against the big account.
Highly dynamic as running the short FAHV quickly changes numbers. As you noted, the FDS= state is volatile. [Edit 4 times, last edit by SekeRob* at Jul 14, 2017 9:15:37 AM] |
||
|
|
|