Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 
World Community Grid Forums
  Category: Active Research
  Forum: Africa Rainfall Project
  Thread: Work Available

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 3596
Posts: 3596   Pages: 360   [ Previous Page | 220 221 222 223 224 225 226 227 228 229 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 5935731 times and has 3595 replies Next Thread
alanb1951
Veteran Cruncher
Joined: Jan 20, 2006
Post Count: 1334
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available
Adri,

Thanks for that list -- the highest work unit number I got was 188713323 - here's what my wingman tracker had to say about it...

Task ARP1_0032951_135_1 was returned by Zen3700 at 2022-10-20T13:04:08+0000:
  Work-unit 188713323 created 2022-10-17T23:54:18+0000 
  Sent date 2022-10-19T22:33:59+0000,  deadline 2022-10-25T22:33:59+0000.

Note that it had been available to the feeder for nearly 24 48 hours :-)

2022-10-20T07:09:30  188826066 ARP1_0027266_139_0

I had a sift through MCM1 and OPN1/G tasks looking for work unit numbers higher than your last ARP1 unit (as above) and the nearest I came up with was this one...

Task MCM1_0191583_2845_1 was returned by Serenity2 at 2022-10-19T09:22:15+0000:
  Work-unit 188828948 created 2022-10-18T05:17:04+0000 
  Sent date 2022-10-18T19:46:36+0000,  deadline 2022-10-24T19:46:36+0000

(Serenity2 and Zen3700 are two of my systems -- I collect the data [for the first time] on the day I return the associated result...)

So it looks as if the entire "batch" was generated over about 13 or 14 hours, and it took somewhere around 40 hours from the start of unit creation to ship them all out... It's not definitive, but it's as close as I can get with the data to hand - thanks again...

Cheers - Al

P.S. I use the new API (as used by the revised web site) and it does expose the work unit creation date if you fetch the WU data -- it has to, as it publishes it to the web page :-)

[Edit: corrected the generation duration estimate -- 8+5.25 does not equal 16 :-)...]

[Second edit: adriverhoef pointed out that I got another bit of arithmetic wrong - 2 days instead of one...]
----------------------------------------
[Edit 2 times, last edit by alanb1951 at Nov 5, 2022 9:56:07 PM]
[Nov 5, 2022 2:59:47 AM]   Link   Report threatening or abusive post: please login first  Go to top 
alanb1951
Veteran Cruncher
Joined: Jan 20, 2006
Post Count: 1334
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available
For anyone who hasn't seen the [revised] initial post in the News thread 2022-11-04 Update (ARP units & Device Manager issues) there's an explanation of why the ARP1 work dried up...

Cheers - Al.
[Nov 5, 2022 3:32:36 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Unixchick
Veteran Cruncher
Joined: Apr 16, 2020
Post Count: 1313
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available
The ARP are flowing again ! Boinc worked perfectly the 3 ARP took 22 minutes to download with all the errors, and then it paused the MilkyWay and started working on ARP. I wasn't home to babysit the machine, so it was a nice surprise to come home too. Let's home they can keep them flowing!
[Nov 5, 2022 4:06:52 AM]   Link   Report threatening or abusive post: please login first  Go to top 
sptrog1
Master Cruncher
Joined: Dec 12, 2017
Post Count: 1593
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available
I received an ARP unit yesterday and iy went thru the transfer process automatically.
I thought there was a trhead to report last received but I cannot find it.
[Nov 5, 2022 5:42:40 AM]   Link   Report threatening or abusive post: please login first  Go to top 
adriverhoef
Master Cruncher
The Netherlands
Joined: Apr 3, 2009
Post Count: 2356
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available
Al:
P.S. I use the new API (as used by the revised web site) and it does expose the work unit creation date if you fetch the WU data -- it has to, as it publishes it to the web page :-)
You are correct!
Sorry, I was referring to the 'wcgresults' part of the API (where you use your verification code to download your own results only),
while the 'wcgstats' part of the API - the new API part - needs your credentials (i.e. you must be logged in) to show you the bits and pieces of workunits to which you contributed (and all your other statistics, such as the characteristics of your devices).

PS. You said in your post, when mentioning task ARP1_0032951_135_1, "Note that it had been available to the feeder for nearly 24 hours :-)". Sorry, but didn't you mean 48 hours? (created 2022-10-17T23:54:18, sent 2022-10-19T22:33:59, returned 2022-10-20T13:04:08)

PPS. When you say that you "collect the data [for the first time] on the day I return the associated result...)", does that mean that you don't wait for your wingmen to return their result or is there a second time (and maybe a third time etc.) that you collect the data? (I was also thinking that your wingmen may experience an error on their side and a new task would get issued.)
[Nov 5, 2022 9:13:07 AM]   Link   Report threatening or abusive post: please login first  Go to top 
alanb1951
Veteran Cruncher
Joined: Jan 20, 2006
Post Count: 1334
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available
Adri,

Firstly thanks for this:
Sorry, but didn't you mean 48 hours?
Yup -- thank you!!! I've gone back and edited the post (with an acknowledgment...)

Secondly, on data collection (per your PPS...): I collect date and time statistics and the status of wingmen (including the Error Log), so I keep checking until all the results I returned on a given day have no remaining wingman tasks outstanding. It's all done in Python, and I build up JSON files of completed result data to avoid hammering the API for data I already have (such as already-finished work units and wingmen's final Error Log content...) - once all data is finalized it is used to update a SQLite database...

I started doing this for ARP1 and MCM1 during the run-up to the shut-down, as I was interested in how many tasks were being returned as non-success results and what the reasons were -- I added OPN1/OPNG later on. It's a little concernng how many retries are due to No Reply or Not Started by Deadline, and interesting watching the ebb and flow of "wrong size" download errors... :-)

Cheers - Al.
[Nov 5, 2022 10:22:09 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Crystal Pellet
Veteran Cruncher
Joined: May 21, 2008
Post Count: 1413
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available
I don't get any ARP-task to run, because all tasks get download or checksum error finally resulting in:

[World Community Grid] No tasks sent
[World Community Grid] No tasks are available for Africa Rainfall Project
[World Community Grid] No tasks are available for the applications you have selected.
[World Community Grid] This computer has finished a daily quota of 1 tasks

So I have to wait another night to maybe get 1 task that probably will fail again on download sad

6-11-2022 07:25 [World Community Grid] Temporarily failed download of arp1_image02_7.32.tga: transient HTTP error
6-11-2022 07:25 [World Community Grid] Temporarily failed download of arp1_image04_7.32.tga: transient HTTP error
6-11-2022 07:25 [World Community Grid] Temporarily failed download of arp1_image08_7.32.tga: transient HTTP error
6-11-2022 07:28 [World Community Grid] Checksum or signature error for arp1_image04_7.32.tga
6-11-2022 07:28 [World Community Grid] Checksum or signature error for arp1_image02_7.32.tga
6-11-2022 07:28 [World Community Grid] Checksum or signature error for arp1_image08_7.32.tga
6-11-2022 07:32 [World Community Grid] Temporarily failed download of MCM1_0192042_7069_MCM1_0192042_7069.txt: transient HTTP error
6-11-2022 07:32 [World Community Grid] Temporarily failed download of MCM1_0192040_7262_MCM1_0192040_7262.txt: transient HTTP error
6-11-2022 08:37 [World Community Grid] Temporarily failed download of MCM1_0192040_8656_MCM1_0192040_8656.txt: transient HTTP error
6-11-2022 09:40 [World Community Grid] Temporarily failed download of arp1_image04_7.32.tga: transient HTTP error
6-11-2022 09:40 [World Community Grid] Temporarily failed download of arp1_image08_7.32.tga: transient HTTP error
6-11-2022 09:40 [World Community Grid] Temporarily failed download of ARP1_0000241_134_ARP1_0000241_134.input: transient HTTP error
6-11-2022 09:41 [World Community Grid] Temporarily failed download of 3d15df74f8e5db2b34f16f0bea24882a.: transient HTTP error
6-11-2022 09:41 [World Community Grid] Temporarily failed download of 7aaa52918fb25d81a94bd27388cd7a5d.: transient HTTP error
6-11-2022 09:41 [World Community Grid] Temporarily failed download of e77ab732211c6e59adf79b5a456475ae.: transient HTTP error
6-11-2022 09:41 [World Community Grid] Temporarily failed download of 673d50ccb5cdafb165e7b57a43c84497.7z: transient HTTP error
6-11-2022 09:41 [World Community Grid] Temporarily failed download of 685bbd74f04871940bb6c35ee75250f3.7z: transient HTTP error
6-11-2022 09:41 [World Community Grid] Temporarily failed download of 56c235811bf9a3a00b6ebaa6756df00f.7z: transient HTTP error
6-11-2022 09:43 [World Community Grid] Checksum or signature error for arp1_image04_7.32.tga
6-11-2022 09:44 [World Community Grid] Checksum or signature error for arp1_image08_7.32.tga
6-11-2022 09:46 [World Community Grid] Temporarily failed download of e77ab732211c6e59adf79b5a456475ae.: transient HTTP error
6-11-2022 09:46 [World Community Grid] Temporarily failed download of ce5d2bc52b6aa6e4340442ad23a93ea7.: transient HTTP error
6-11-2022 09:51 [World Community Grid] Temporarily failed download of wcgrid_arp1_wrf_7.32_windows_intelx86: transient HTTP error
6-11-2022 09:51 [World Community Grid] Temporarily failed download of wcgrid_arp1_gfx_7.32_windows_intelx86: transient HTTP error
6-11-2022 09:55 [World Community Grid] Checksum or signature error for wcgrid_arp1_gfx_7.32_windows_intelx86
6-11-2022 09:55 [World Community Grid] Temporarily failed download of 0df57141f48d54f6d10b42b00bc6b5d0.7z: transient HTTP error
6-11-2022 09:59 [World Community Grid] Temporarily failed download of arp1_image04_7.32.tga: transient HTTP error
6-11-2022 09:59 [World Community Grid] Temporarily failed download of arp1_image08_7.32.tga: transient HTTP error
6-11-2022 09:59 [World Community Grid] Temporarily failed download of wcgrid_arp1_wrf_7.32_windows_intelx86: transient HTTP error
6-11-2022 09:59 [World Community Grid] Temporarily failed download of wcgrid_arp1_gfx_7.32_windows_intelx86: transient HTTP error
6-11-2022 09:59 [World Community Grid] Temporarily failed download of ARP1_0029898_134_ARP1_0029898_134.input: transient HTTP error
6-11-2022 09:59 [World Community Grid] Temporarily failed download of dea7f7df7d68d4bd41ece569f9895e80.: transient HTTP error
6-11-2022 09:59 [World Community Grid] Temporarily failed download of 86087d2594ca292dfe2ed24490e2e15d.: transient HTTP error
6-11-2022 09:59 [World Community Grid] Temporarily failed download of 6a9880126a68922dab747e576b59ad45.: transient HTTP error
6-11-2022 09:59 [World Community Grid] Temporarily failed download of 12e3de18a99017a8b2d1381cb412fd2f.: transient HTTP error
6-11-2022 09:59 [World Community Grid] Temporarily failed download of cfaff5fabff9c6f8f352ba6258a31322.7z: transient HTTP error
6-11-2022 09:59 [World Community Grid] Temporarily failed download of 42632ffbd68412d4c336c7a45064d86c.7z: transient HTTP error
6-11-2022 10:01 [World Community Grid] Checksum or signature error for arp1_image04_7.32.tga
6-11-2022 10:02 [World Community Grid] Checksum or signature error for arp1_image08_7.32.tga
6-11-2022 10:03 [World Community Grid] Temporarily failed download of arp1_image08_7.32.tga: transient HTTP error
6-11-2022 10:03 [World Community Grid] Temporarily failed download of wcgrid_arp1_wrf_7.32_windows_intelx86: transient HTTP error
6-11-2022 10:03 [World Community Grid] Temporarily failed download of ARP1_0000939_134_ARP1_0000939_134.input: transient HTTP error
6-11-2022 10:03 [World Community Grid] Temporarily failed download of e727392f21f15088a993a591bbb82657.: transient HTTP error
6-11-2022 10:03 [World Community Grid] Temporarily failed download of da32496c8f13d9a2c63778f2a68ff5bf.: transient HTTP error
6-11-2022 10:03 [World Community Grid] Temporarily failed download of 39a5055e1bbd2873c95ac66645f6c638.: transient HTTP error
6-11-2022 10:03 [World Community Grid] Temporarily failed download of 9d8ecbb71194ab453761467ad9c9c35c.7z: transient HTTP error
6-11-2022 10:03 [World Community Grid] Temporarily failed download of 03aad3b0192e98169a31f6374fc3d498.7z: transient HTTP error
6-11-2022 10:03 [World Community Grid] Temporarily failed download of ARP1_0000939_134_ARP1_0000939_input_d01: transient HTTP error
6-11-2022 10:06 [World Community Grid] Checksum or signature error for arp1_image08_7.32.tga
6-11-2022 10:08 [World Community Grid] Temporarily failed download of arp1_image08_7.32.tga: transient HTTP error
6-11-2022 10:08 [World Community Grid] Temporarily failed download of wcgrid_arp1_wrf_7.32_windows_intelx86: transient HTTP error
6-11-2022 10:08 [World Community Grid] Temporarily failed download of ARP1_0008944_134_ARP1_0008944_134.input: transient HTTP error
6-11-2022 10:08 [World Community Grid] Temporarily failed download of df4657bd6b81888238708196fe32936a.: transient HTTP error
6-11-2022 10:09 [World Community Grid] Temporarily failed download of 0e38ddd009ec2131eafb702e2413c177.: transient HTTP error
6-11-2022 10:09 [World Community Grid] Temporarily failed download of 3801a04b8d0a3be1ee14757d08a8063d.: transient HTTP error
6-11-2022 10:09 [World Community Grid] Temporarily failed download of 1066bf39835774e6d5a4cbdfe327a269.: transient HTTP error
6-11-2022 10:09 [World Community Grid] Temporarily failed download of 6db9b3f370e1ba1eb08019cf7b61408a.7z: transient HTTP error
6-11-2022 10:09 [World Community Grid] Temporarily failed download of 05c6e9278588c013844bc147c8612da2.7z: transient HTTP error
6-11-2022 10:09 [World Community Grid] Temporarily failed download of d7f44ca3d7ffb0ffb5ca4343b7eacc5e.7z: transient HTTP error
6-11-2022 10:09 [World Community Grid] Temporarily failed download of ARP1_0008944_134_ARP1_0008944_input_d02: transient HTTP error
6-11-2022 10:09 [World Community Grid] Temporarily failed download of ARP1_0008944_134_ARP1_0008944_input_d01: transient HTTP error
6-11-2022 10:09 [World Community Grid] Temporarily failed download of ARP1_0008944_134_ARP1_0008944_input_d03: transient HTTP error
6-11-2022 10:18 [World Community Grid] Checksum or signature error for arp1_image08_7.32.tga
6-11-2022 10:18 [World Community Grid] Temporarily failed download of 0e38ddd009ec2131eafb702e2413c177.: transient HTTP error
[Nov 6, 2022 10:51:59 AM]   Link   Report threatening or abusive post: please login first  Go to top 
adriverhoef
Master Cruncher
The Netherlands
Joined: Apr 3, 2009
Post Count: 2356
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available
Al:
I've gone back and edited the post (with an acknowledgment...)
Ha, thanks! smile

I keep checking until all the results I returned on a given day have no remaining wingman tasks outstanding.
A question that you could expect wink - how often do you check?

It's all done in Python, and I build up JSON files of completed result data to avoid hammering the API for data I already have (such as already-finished work units and wingmen's final Error Log content...) - once all data is finalized it is used to update a SQLite database...
That's very admirable as I don't speak Python. biggrin

It's a little concernng how many retries are due to No Reply or Not Started by Deadline, and interesting watching the ebb and flow of "wrong size" download errors... :-)
I'm sure the techs and the project scientists are also interested in this matter.

Adri
---
I don't dance, so it takes two other people to tango.
[Nov 6, 2022 1:04:27 PM]   Link   Report threatening or abusive post: please login first  Go to top 
alanb1951
Veteran Cruncher
Joined: Jan 20, 2006
Post Count: 1334
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available
Adri,
A question that you could expect wink - how often do you check?
Usually once a day but occasionally twice[1] -- because ongoing processing depends on how many items appear each time I sample, it's currently a manual process; normally once a day is often enough to avoid missing any unit validations, but if I'm monitoring specific tasks I might sample more often (or, more likely, just use the web site or "Swagger UI" for intermediate checks...)

The manual process is the result of a simple "per day" initial script growing into something far more complex as I found other things of interest to add (but remaining "per day") -- it has the advantage that I get to see "sudden" changes, but has obvious disadvantages too (such as time spent)... :-)

Now that I have a far better idea of what I need from it, I may re-script it from scratch using a different structure that automates better if/when WCG eventually stabilizes enough to no longer require baby-sitting and constant tuning of [cross project] work-loads! That might be able to sample more frequently without flooding my file-store with script output :-)

Cheers - Al.

[1] Yes - I should've put that in the original message :-)
[Nov 6, 2022 10:36:00 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Mike.Gibson
Ace Cruncher
England
Joined: Aug 23, 2007
Post Count: 12594
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work Available
Sunday Report .

Only 781 units validated in 7 days as no new units were released for the first 6. So a very low average of 111.6 per day.

Assuming that a full generation 182 will be the last, there are 1,766,047 units still outstanding, so my forecast end date would now be 28 December 2025, however we are still coming out of testing so we should finish well before then.

Sone of the extremes have moved in the last day.

The definition of extreme has now caught up.

There are 46 Extremes and 17 Accelerated units, although the numbers in their generations are 220 & 3,593 due to lack of movement.

The situation with downloading units is still dire.

Mike
[Nov 7, 2022 6:21:29 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 3596   Pages: 360   [ Previous Page | 220 221 222 223 224 225 226 227 228 229 | Next Page ]
[ Jump to Last Post ]
Post new Thread