| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Active Research
Forum: Africa Rainfall Project
Thread: Work Available |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 3520
|
|
| Author |
|
|
Mike.Gibson
Ace Cruncher England Joined: Aug 23, 2007 Post Count: 12564 Status: Offline Project Badges: 
|
Thank you, Crystal Pellet.
Another earlyish one and the results being returned seem to have stabilised around 16,000 per day. 081 indicates we are at about 44.3%, but for this month I am assuming 2 generations behind to allow for the stragglers, so 43.2%. The latest interval is just 3.46619 days and the 10-interval average is down to 4.50010 days. The end date forecast is now October 2022, however, if this rate continues, the forecast would be May 2022. I would expect the next generation to start about 30 July. Mike |
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges: 
|
We are currently sitting with a backlog of about 2,500 results available to be sent. We need to run about 800 more results per day in order to eliminate that backlog.
----------------------------------------In answer to some of @entity's questions: The average turnaround for a "standard" job for ARP1 is 2.3 days while the average turnaround for a "high priority" job is 1.7 days. I have just made the change to reduce the definition of a reliable host to be one that returns the result within 2.0 days (down from 2.5 days) which will likely result in a bigger difference between these two values over the next 3-4 days. I can't go lower because we risk have too few hosts qualifying for reliable (which then results in work not being sent out to anyone except users with reliable hosts). One thing to note. The "spread" between the leading generation and the trailing generation is not a significant issue until the very end of the project. It is much more important to the researchers to run the project as fast as we can. Right now I'm running the 5 leading generations at normal priority and these represent about 18.5% of the results that were received over the past 24 hours. I will be checking this periodically and if it goes about 20%, I will change it so that the leading 6 generations run at normal priority (and so on to keep the 20% as the upper limit). I'm doing this to manage the spread to a reasonable amount without impacting the speed at which the project runs. [Edit 1 times, last edit by knreed at Jul 26, 2021 8:13:18 PM] |
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges:
|
And also - thanks @entity to adding those extra 200 cores!
|
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Thanks Kevin for the information.
That 18.5% stat is an interesting stat. I would have thought it would have been much, much higher. That seems to indicate that more than 80% of the returned work is the high priority work. Unfortunately for me, due to an existing error in the boinc client, I'm probably going to lose my reliable status due to having too much work in my queue that needs to be worked off. I think I can get around the client issue by using the <fetch_minimal_work> flag and return to reliable status in the next few days. |
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges:
|
That 18.5% stat is an interesting stat. I would have thought it would have been much, much higher. That seems to indicate that more than 80% of the returned work is the high priority work. Sigh - or it means I reversed the info. 81.5% is work being returned at "normal"priority vs 18.5% being returned is "high priority". I plan to watch to make sure that "high priority" work doesn't exceed 20%. Sorry for the mis-statement. |
||
|
|
Mike.Gibson
Ace Cruncher England Joined: Aug 23, 2007 Post Count: 12564 Status: Offline Project Badges:
|
I presume that you are referring to only one generation for each of the stragglers or the current generations. If so, that ties in with my figures.
The proportion is likely to fluctuate up and down as we move through the generations. Adding a new generation will reduce the proportion of priority cases. Clearing out an early generation will not actually reduce the number as that just increases the number outstanding in the next generation. I would agree that 20% priority units is probably about right and that can be juggled by adjusting the number of generations regarded as 'normal' units. Resends should be regarded as 'priority' as usual because crunchers get frustrated seeing mounting units 'awaiting validation'. Reducing the number of machines qualifying for priority units is not the best way to clear them. Asking crunchers to reduce their cache would increase the number of stragglers being cleared by increasing the number of machines qualifying for them. A large cache is no longer needed to maintain a steady workload, but old habits die hard. Mike |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
That 18.5% stat is an interesting stat. I would have thought it would have been much, much higher. That seems to indicate that more than 80% of the returned work is the high priority work. Sigh - or it means I reversed the info. 81.5% is work being returned at "normal"priority vs 18.5% being returned is "high priority". I plan to watch to make sure that "high priority" work doesn't exceed 20%. Sorry for the mis-statement. Ah! -- That is more in line of what I was thinking it should be. Now though, I'm totally confused as to what the goal is that you are trying to accomplish. I moved my cores over in an effort to help clear the stragglers (high priority work). Now I'm understanding that you are trying to limit the high priority work to ~20% of the daily returned results. Which, based on your previous statement, is where it was at essentially (81.5% normal priority, 18.5% high priority), give or take. There is only a one and one half percent difference which, based on 16000 WUs per day, is only 240 additional high priority WUs per day to hit the 20% As cores are moved over, instead of helping clear high priority work, they will be used to work normal priority because that additional 240 would go quickly, That would suggest normal work will complete quicker (more cores working normal work) and the gap between leading and trailing work widens. My reasoning is based on the belief that you are not changing the amount of high priority work done in a given day by any appreciable amount but potentially increasing the normal work done by a wide margin (based on the number of additional cores added to the computational landscape). I'm starting to think you might not need the additional cores. Thinking back to the beginning of the project, it seems like this project was always limited to about 15,000 to 16,000 WUs per day. That seemed like a "sweet spot" where the project could move forward but didn't overwhelm the backend with data. This seems to be where you are now. Has that backend data issue been resolved? Does this project still have a soft limit on the amount of work given out per day? Maybe this is one of those, "be careful what you ask for..." moments |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Started getting these messages now...
----------------------------------------Anyone else getting these? Wed 28 Jul 2021 11:51:09 AM CDT | World Community Grid | update requested by user Wed 28 Jul 2021 11:51:11 AM CDT | World Community Grid | Sending scheduler request: Requested by user. Wed 28 Jul 2021 11:51:11 AM CDT | World Community Grid | Requesting new tasks for AMD/ATI GPU Wed 28 Jul 2021 11:51:12 AM CDT | World Community Grid | Scheduler request completed: got 0 new tasks Wed 28 Jul 2021 11:51:12 AM CDT | World Community Grid | No tasks sent Wed 28 Jul 2021 11:51:12 AM CDT | World Community Grid | No tasks are available for Africa Rainfall Project Wed 28 Jul 2021 11:51:12 AM CDT | World Community Grid | No tasks are available for Help Stop TB Wed 28 Jul 2021 11:51:12 AM CDT | World Community Grid | Tasks for Intel GPU are available, but your preferences are set to not accept them Wed 28 Jul 2021 11:51:12 AM CDT | World Community Grid | Project requested delay of 121 seconds UPDATE: They have returned [Edit 1 times, last edit by Former Member at Jul 28, 2021 6:12:03 PM] |
||
|
|
MJH333
Senior Cruncher England Joined: Apr 3, 2021 Post Count: 274 Status: Offline Project Badges:
|
Hello Mike, I've got an 082:
ARP1_0005783_082 Cheers, Mark |
||
|
|
Mike.Gibson
Ace Cruncher England Joined: Aug 23, 2007 Post Count: 12564 Status: Offline Project Badges:
|
Thank you, Mark.
Another early one and the results being returned have increased to nearer 18,000 per day. At that rate it would take a complete generation 4 days. 082 indicates we are at about 44.8%, but for this month I am assuming 2 generations behind to allow for the stragglers, so 43.7%. The latest interval is just 2.27367 days and the 10-interval average is down to 4.382005 days. The end date forecast is again October 2022, however, if this rate were to continue, the forecast would be April 2022, but I doubt it. I would expect the next generation to start about 31 July. Mike |
||
|
|
|