| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
| World Community Grid Forums
|
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 42
|
|
| Author |
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
I know WCG has the statistics about which machines tend to return results within a 24 hr period on a continual basis.
It would likely be beneficial if there was a separate queue of WU's for each project that are specifically for machines that return the WU's within 24 hrs and dispatch those WU's only to the machines returning results very fast. Leave the default return date of 12 days / 14 day / 3 months as it is now don't change that. It may not benefit the scientist, but it would reduce the backlog of validated wu's for those that are always waiting for others to return the WU's. As it is right now; I've got a backlog of 45 WU's (and growing) that need validation. Getting the WU's validated as soon as possible using machines that have same day response, also has the opportunity to change the overall complexion of the WGC accomplishment graphs which are produced. Anyway, just my thoughts. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Here is knreed's most recent post in this area: http://www.worldcommunitygrid.org/forums/wcg/viewthread?thread=25998#236316
Lawrence |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Lawrence,
----------------------------------------Thank you for the link in your reply. After reading the information in the link, it appears I failed to be as clear and precise in my prior post as I really needed to be. Please permit me to try again with my position. The observation was not for any particular project as I believe the post you referenced presented. In my post, I had intended to supply sufficient information where the methodology, not necessarily a technique, could be designed and possibly implemented such that all WU's could fall into separate queues specifically to be dispatched to machines which return WU results within a 24 hour period. This method could easily be used for high priority dispatching of WU's within a specific queue of WU's for a particular project. There appears to already be a mechanism in place for the express purpose of having the necessary WU's and the requisite validation WU's to be sent out for immediate processing forming quick turnaround. My suggestion, while similar is, I believe, just a tad different. As I see it, some set of WU's and their associated validation WU's would all be placed into this queue for nominal WU completion and return for validation to be completed within a 24 hr period. There are some benefits for this type of implementation. Many folks, will stop carrying long queues of WU's that need to be completed. In other words, I suspect many will reduce their number of "Additional Days" of WU depth. As I indicated before, I can only suspect this has an opportunity to change the complexion of how the entire WGC behaves in complete through put of WU's. Here's my oldest WU in pending validation: Here's my most recent WU in pending validation: Now; if there were a queue for machines that returned WU's typically within 24 hrs were available and all the WU's listed in the WU status as shown in the above; I suspect the throughput of the WU's in general would;
Personally, I have my system set up so that it only starts to look for additional work just about the time a WU is about to complete, something like about 7 minutes of WU completion. Does that present a better explanation about what I would like to see? Thanks ---Barney [Edit 1 times, last edit by Former Member at Jul 3, 2009 12:41:45 AM] |
||
|
|
gb009761
Master Cruncher Scotland Joined: Apr 6, 2005 Post Count: 3010 Status: Offline Project Badges:
|
Personally, I think the crux of the matter is - being patient...
----------------------------------------At times, we've all been eager to get x number of WU's validated (for whatever reason - although I'm guessing, mainly for "The Badge"), yet, once a WU has gone into PV status, I think it's very rare indeed for a WU to not be validated at some stage in the not too distant future. Just from a technical point of view (and please, those who have far more knowledge/experience than I do on this matter, please feel free to jump in and correct me), the reissuing of WU's out to the 'reliable' computers, is done automatically, for a certain set criteria (e.g., a 'No Reply'). If the techs are going to have to figure a way of generating yet another series of 'quick return' queues, how many 'hoppers' are they going to have to ensure are filled? We're already having the odd occasional 'empty hopper' situation - just think what it'd be like when that number of hoppers were doubled over night... As we've seen from other threads, there are hoppers for each platform, project, CPU chip-set etc., and therefore, for 1 project (of which, as you know, we've got 9 currently up and running), there could be a dozen or so 'hoppers'... So in summary, I think it's a nice idea, although rather impracticable - especially when all that's needed, is a bit of patience ![]() Edit: Also, isn't there then a danger, that, due to the volume of WU's requiring a 'reliable computer', that they themselves get flooded out with WU's and become 'unreliable' - therefore, defeating the purpose? ![]() [Edit 1 times, last edit by gb009761 at Jul 3, 2009 1:14:55 AM] |
||
|
|
Sid2
Senior Cruncher USA Joined: Jun 12, 2007 Post Count: 259 Status: Offline Project Badges:
|
Personally, I think the crux of the matter is - being patient... So in summary, I think it's a nice idea, although rather impracticable - especially when all that's needed, is a bit of patience ![]() If the system can reward the quick turnaround users with Betas. . . why couldn't the same system muster those same users together to get work turning faster? I believe the crunching experience is enhanced with work turning in two days instead of two weeks. ![]() |
||
|
|
Greg Lyke
Advanced Cruncher Joined: May 30, 2008 Post Count: 50 Status: Offline |
I think that the idea is a good one, but like gb said, not really practical.
What I would be interested to see is how many people (as a percentage of users) have a full 10 days worth of projects in their queue. And then of those, how many of them connect to the internet every 1-3 days (or are constantly connected). If they are connecting fairly often then they really don't need a filled up 10 day queue do they? If there aren't that many people using the full 10 days who really need that amount of time, perhaps lowering the buffer down to 7 days is something to try. Another alternative is only allowing a maximum of two days worth of work units per project at any one time (so if you wanted a 10 day queue buffer you would have to choose 5 different projects). While some people would be waiting/wingmen for those projects that are crunched later in the 10 day cycle, the total number of wingmen waiting should be reduced. This is because while days 1-2 of (for example) CEP subset 789 would be filled, he wouldn't be filling days 3-10 with 789's, those would be going out to different users (who may or may not have a fast turnaround). Some people would still have to wait for him to finish, but there wouldn't be as many waiting as in the current system. Or you could just be patient & wait for everything to shake out in the end as it does now.... |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Or take a trick from GPU grid. They give a 25% point bonus if the WU is returned within 24 hours... might not make everyone lower their buffer, but my buffer would definitely drop to less than 24 hours.
----------------------------------------![]() and I have a feeling it would make enough people change to make a noticeable impact... [Edit 1 times, last edit by Former Member at Jul 3, 2009 5:12:01 AM] |
||
|
|
Sid2
Senior Cruncher USA Joined: Jun 12, 2007 Post Count: 259 Status: Offline Project Badges:
|
Or take a trick from GPU grid. They give a 25% point bonus if the WU is returned within 24 hours... might not make everyone lower their buffer, but my buffer would definitely drop to less than 24 hours. ![]() and I have a feeling it would make enough people change to make a noticeable impact... ![]() ![]() |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Otis11
----------------------------------------and I have a feeling it would make enough people change to make a noticeable impact... And that in and of itself is the real crux of the matter. As my images above easily demonstrated; its not about "being patient" it certainly revolves around the demonstrable benefit in WU throughput. It's not about the badges, it is about getting WU's:
So those with the big brains that can use the information for the purpose of mankind. (that as it is these days). Seemingly it does not benefit anyone for WU's that are completed within hours of initial dispatching sitting and awaiting validation by others for 2+ weeks. Awaiting validation within a 24 hour period seems to be reasonable. WCG must know which machines return WU's within the shortest time periods. So creating a specialized queue containing the entire WU batch and putting that entire batch within the "fast Queue" seems a reasonable and responsible thing to request for numerous reasons. Now, I'm certainly not suggesting that if machines suddenly start moving over, they be immediately placed into list of fast responders. This is something that requires a consistent track record, I don't know, say 10 consecutive days of returning results within 24 hrs? Turn your machine off whilst you go on holiday for say a week; and you must again certify your machine as a fast responder? These are just thoughts and not truly intended to be implementation perspectives. The masters that run WCG have much better insight to this than I, a simple mortal. When we had single core systems, that could hardly get a WU returned in 24 hours to 48 hours, the 7 / 12 / 14 / day return validation methodology appears to have been a sound and reasonable decision which ensured the most WU's were returned completed which eliminated copious re-work of WU's. Today, many are returning WU results using unbelievably fast, multi-core CPU's tied to systems with almost incomprehensible quantities of memory; HDD space; and actual CPU processing capacity not available only a few years prior. These technological advances have been the single biggest contributor to being able to return many WU's within a 24 hour period of time. In my mind's eye, as it relates to not having a "queue for 24 hour turnaround" seems strikingly similar to one justifying using a howitzer as the appropriate tool to kill unwanted mice. ![]() As an aside, while I can't say this with any real certainty, one could argue the point, should WCG implement a "Fast Queue" response for each and every project, additional projects regardless of size could benefit in reduced time to completion of the complete project. BTW, I just looked at my PV queue. When I started this thread, I was at a PV queue depth of 45. I'm quickly approaching a PV queue depth of 60. Should the results continue to pile up like this, some possibility exists my simple little machine could have a PV queue depth of over 100 before long. Surely WCG can do something about getting those of us in PV purgatory out of this mess. Wouldn't it be simply amazing if by the adoption and implementation of this seemingly simple change, a project could get all it's WU's performed say 6 / 7 months earlier than initially anticipated and a cure for something could be found in time to save someone that would have otherwise been lost? Dreamy eyed, Pie in the Sky, perhaps... but what if it worked out to be the one it saved was you, one of yours or one of your friends, would it still be "impracticable?" So wadaya say WCG guys can you do something to encourage a seemingly simple behavioral change within the community? (Hey Camera guy, you gettin' all this? Opps sorry, I thought this was a Sham-Wow Commercial! )[Edit 4 times, last edit by Former Member at Jul 3, 2009 8:42:49 AM] |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Need more input XJ25?
----------------------------------------Batches are generally not moved through the assimilator until 100% complete. Just a few 'reliable' devices to fall over and the gain is out the window, unless the deadlines are reduced accordingly. What's a batch to a scientist if he needs a series of batches in an experiment? At > 90% already returned within 4 days (many moons ago it said 96%), the relative gain is minimal for WCG and as knreed wrote a few months ago, eventually squeezing the last 0.1% out becomes a disproportional effort. Disk storage is cheaper. What's moved to the assimilators is removed from the public facing database, so better look quickly on your RS page. Other projects you frequently don't see completed work beyond 24 hours, here you see it for about 4 days after validation, at least and most of the time. Separate, How much work should be allocated to 'reliable' and where does the work go of the normal, no rush crunchers' repair no reply? Already there was a backlog developing for those tasks, so 'reliable' was increased to at least 2 days... that is, with my 2 day cache and rDCF of 1.5 am still getting them. Initial rush deadlines are now 40% of the original, speak 4 days for most projects. Oh and Sid2, no quick returner is rewarded with Beta's. That's been a very long misunderstanding. Every client enrolled and contacting the feeders for more work at the right moment gets them! If there is one or the other FAQ around still saying so, let me know which and it will be amended.
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
|