Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
World Community Grid Forums
Category: Beta Testing Forum: Beta Test Support Forum Thread: Better spread of beta WUs among beta testers |
No member browsing this thread |
Thread Status: Active Total posts in this thread: 18
|
Author |
|
F. Bravo
Cruncher Joined: Apr 28, 2007 Post Count: 13 Status: Offline Project Badges: |
Dear WCG techs,
----------------------------------------As I've told in the "New Beta Test - July 21, 2017 [ Issues Thread ]", I'm running WCG on a 64 bits Windows 10 notebook, but I haven't received one single WU for this last ongoing beta test. On the other hand, I see many users (in this Forum) saying that they received 15 WUs, 10 WUs etc. I think WCG should find a way to spread beta WUs more efficiently among users. I've been running WCG 24/7 for about 10 years and of course, I've already run some beta WUs. Nowadays I'm keeping a 0,10 days cache, in order to try to receive some beta WUs. However, I see that their distribution is extremely unequal. Besides, I think that the more different machines test beta WUs, techs will have more information about WUs' different behaviors in diverse configurations. Therefore, I'd like to kindly request that you recognize this problem and try to solve it, not only for the ongoing beta test, but also for the future ones. I'm speaking not only on my behalf, but also on the behalf of the other users who are eager to participate in beta tests and are not able to do so. Yours sincerely, F. [Edit 1 times, last edit by F. Bravo at Jul 30, 2017 7:03:28 PM] |
||
|
gb009761
Master Cruncher Scotland Joined: Apr 6, 2005 Post Count: 2977 Status: Offline Project Badges: |
F. Bravo, although I've no evidence to go on, it could well be that those users who, as you mention, have received 10/15 WU's, could have several machines with upwards of dozens of cores available to them - thus, the more cores they've got available, the higher the chances of them being able to pick them up.
----------------------------------------Another aspect of it, is when (as we currently do), we have projects such as HST & FAH2 where there's a limited supply of WU's available, people can simply select these projects knowing that, when their BOINC sessions request work, the only WU's that they'll pull back, are Beta WU's. Thus, if they're "in the right place at the right time", they can set their machines up so as to raise their chances significantly for getting some Beta WU's. I don't know when you joined the Beta programme (and I can't just recall when it changed), but originally there were no restrictions at all as to how many Beta WU's could be pulled down for one particular core. Nowadays, a limit of "one Beta WU/Core" is set - thus helping to spread the work units around a lot more. PS. I, too, am in the Beta programme, but (as yet), haven't received any Beta WU's this time around. What doesn't help my cause, is that I've currently got SCC set as a project - thus, my buffers are constantly being topped up on their first request. |
||
|
F. Bravo
Cruncher Joined: Apr 28, 2007 Post Count: 13 Status: Offline Project Badges: |
Dear gb009761,
Thanks for you answer; I think you are right. I see that the new "rule" of "one Beta WU/Core" helps to make the system fairer. However, it's undeniable that the fairest would be a system of "one Beta WU/user". Of course that, once the WUs are spread following this rule (one WU/user), users who have already received one WU could receive another one, and so on. Besides, as I noted before, it would also be better for the whole WCG beta system. Like I said, "the more different machines running beta WUs, techs will have more information about WUs' different behaviors in diverse configurations." If a machine has 15 cores, it may receive 15 WUs at once, but all in the same machine and configuration; it means less configurations and less diversity for the techs to analyze. More users = more diversity = better system. |
||
|
gb009761
Master Cruncher Scotland Joined: Apr 6, 2005 Post Count: 2977 Status: Offline Project Badges: |
F. Bravo, you do have a lot of valid points and I'm sure that there are numerous different ways a fairer system could be brought about (that is, if WCG have the time/resources and inclination to investigate/implement it).
----------------------------------------There is though one drawback to restricting 1 Beta WU to one user (or even to one machine - no matter as to how many cores it's got), is that the project won't be tested running more than 1 Beta WU on a machine that may have 8, 12 or 16 etc. cores, and thus, any problems this could cause when the project is rolled out with no restrictions, won't be spotted (case in point, CEP2, where running 1 WU didn't cause many/any issues, but running 4 or 8 at a time, generally did). One potential solution, could be to, as well as having the 1 WU/core restriction, could be to restrict that machine to only have 1 WU/core per 24 hrs (or wave of a Beta test). This may/may not have any effect - particularly the 24 hr restriction - as most Beta tests only have WU's available for a very short period, and thus, by the time a WU has been crunched and returned (thus, freeing up a core for another Beta WU), they've probably all been sent out. |
||
|
F. Bravo
Cruncher Joined: Apr 28, 2007 Post Count: 13 Status: Offline Project Badges: |
Dear gb009761,
Thanks for answering. You have a very good point, that I hadn't thought of. You suggested a "1 WU/core per 24 hrs" restriction. However, I think it wouldn't solve the problem, as such distribution system would take away all WUs too fast and by too few users (because some users may have 100 cores available, for example), not leaving chance for more users. I think it would be more or less like it is already. Maybe a better solution would be to allow 4 (or 8) WUs/machine/user per every 24h or 48h (if this is feasible). Therefore, users with more than one machine would receive work in just one of them, but being allowed to crunch as many WUs as allowed in that machine, thus solving the multicore problem you appointed. If after 24h (or 48h) the machine finishes all the WUs and there are more WUs available, the user would be able to crunch more, in that same machine or another (but only one machine at a time). The point is: one user wouldn't be able to crunch 10 or 15 WUs simultaneously and WUs would be more spread (which would also increase diversity, as I said above). Other solutions may also be thought of, and I don't know if the ones we suggested are feasible. However, I think this is a question that must be addressed. Let's try to address it and discuss the possibilities together! I hope the techs will read this thread and join the discussion. |
||
|
gb009761
Master Cruncher Scotland Joined: Apr 6, 2005 Post Count: 2977 Status: Offline Project Badges: |
F. Bravo, after quite a bit of digging around, I've found the announcement from Kevin Reed on the 22nd of March 2010, announcing the 'new' policy (as, back then), of 1 WU/core. The link to it is here Before then, it really was a 'free-for-all' (as can be read in the threads from way back then, with numerous "tips and tricks" to get more Beta WU's). Of course, back then, there wasn't the proliferation of computers with multiple cores as we have now - I only had 2 cores available to me in my 1 laptop, as opposed to the 8 cores I've now got at my disposal.
----------------------------------------Now, before this new policy was implemented, I didn't know what was possible (and I still don't), so a lot really depends on what can/can't be implemented and as to how much resources it'd take to implement (or even, if WCG see this as something they want/need to look into). I also don't know as to; a) how many active members are signed up for the Beta programme (i..e, who's returned a WU in the last month), b) how many cores are available from these active members, and c) how many Beta WU's are made available for any one round of testing (this, of course, will no doubt be a fluid number depending on what they're testing etc.). Thus, as you quite rightly pointed out (and I inferred), limiting Beta Wu's to 1/core/24 hr window may not actually be sufficient to 'spread the load' (i.e. there may be far more cores available than Beta WU's), thus another approach would be needed. One possible way, could be to group possible available Beta cores into groups of (say), 1,000 cores (Beta-A testers), then the next 1,000 into Beta-B testers etc, and target each round at a different group of Beta testers. Of course, this may need some development work form the WCG techs (and of course, as with everything, they may not see this as a high a priority issue compared to the many things on their endless "to do/wish" list). So, really, we can discuss this 'until the cows come home', but, until we get some feedback from the WCG techs, there's not really a lot we can do other than come up with and discuss suggestions. |
||
|
F. Bravo
Cruncher Joined: Apr 28, 2007 Post Count: 13 Status: Offline Project Badges: |
Dear gb009761,
You're right; this discussion will be useless without the techs' participation. However, maybe they'll only pay attention if more people come and comment on this thread. Do you think there's anything else we can do? |
||
|
duanebong
Advanced Cruncher Singapore Joined: Apr 25, 2009 Post Count: 134 Status: Offline Project Badges: |
However, it's undeniable that the fairest would be a system of "one Beta WU/user". It's not possible to achieve a system that is viewed as "fair" by everyone. Fairness depends on each individual, and some could argue that "one Beta WU/user" is not fair at all. Some users have server farms with 100s of high clockspeed cores. They run 24hrs/day, 7 days/week and can return beta WUs within hours. Others have a single low power core and run for a few hours on weekdays only - taking a week to return a single WU. Science output at WGC would be impaired if both are allowed 1 beta WU per user. The current method is already quite meritocratic: 1. Already limited to 1 WU per core. At the same time, we receive more WUs proportionally to the computing resources we dedicate to WGC. It's also an incentive to add multiple / more powerful machines. 2. Machines that run 24/7 complete more WUs (naturally!). So they contact the server more often to report completed work. In doing so, they have a higher chance of finding beta WUs. One can improve chances of getting beta WUs by running 24/7, at the same time WGC also gets more science done - win/win! To make it even more meritocratic: A batch of beta WUs can be broken down into 48 parts and released every 30mins over a 24hr period. (1) reduces the chances one user gets a large cache of beta WUs because he just happened to check the server when a new batch was released. (2) many projects originate from the US, and researchers release new WUs during their working day. Users on the other side of the world miss out if their machines are turned off over night. In my view, tweaks need to maintain the principles of meritocracy. Instead, aim to increase fairness by targeting the "chance" factor of getting a beta WU. [Edit 1 times, last edit by duanebong at Jul 31, 2017 10:12:51 AM] |
||
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
The concept of 'user' is peripheral... the function is only there to add up statistics of results produced by multiple machines registered under the same 'user'. There is zero interaction between clients, none whatsoever, in fact they're agnostic of each other when it comes down to communicating. Just to get 'fair-er' would be a major rewrite, just for exactly what? A badge?
----------------------------------------Distributed across the day... don't make me laugh. Those who want them already scripted their way to further increase their chance, HSTb baring witness to the openly discussed efforts to beat the system. Want more beta work, beat the system, start by subscribing to get an instant mail the moment something is posted in the BETA (Read Only) forum, to then sit ready to hammer the update button, or script to hammer the update button, every n seconds. It wont be recoded for you... beta goes out with absolute feeder priority, and right that is. edits: auto spell corrections corrected. [Edit 2 times, last edit by SekeRob* at Jul 31, 2017 10:30:06 AM] |
||
|
imakuni
Advanced Cruncher Joined: Jun 11, 2009 Post Count: 102 Status: Offline Project Badges: |
Honestly, we just need to increas the volume of WU produced. Taking a look at 2 Betas I was able to snag this wave, they had Min. Replication of 1, and both were run on Windows 10. If I was doing it, I would have AT LEAST 1 Windows 10 use, 1 Win 8, 1 Win 7, 1 Linux and 1 Mac per WU (assumind Mac is supported).
----------------------------------------Afterall, this is BETA. It doesn't really matter if tests are redundant, the purpose is to make SURE that they are working. Want to have an image of yourself like this on? Check this thread: https://secure.worldcommunitygrid.org/forums/wcg/viewthread_thread,29840 |
||
|
|