Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 9
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 75635 times and has 8 replies Next Thread
JmBoullier
Former Community Advisor
Normandy - France
Joined: Jan 26, 2007
Post Count: 3716
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Minimum quorum reduced from 14 to 10 days for the Rice project

Since June 24, 2008 the minimum quorum for getting a Rice WU validated has been reduced from 14 to 10.

This is the most visible of several measures currently taken by the techs to improve the throughput for this project and to get a more homogeneous level of details fot the results regardless of the difficulty of the genes involved.

As a side effect results should be validated sooner since the initial distribution is still 19.
----------------------------------------
Team--> Decrypthon -->Statistics/Join -->Thread
[Jul 1, 2008 7:50:28 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Trong Nguyen
Cruncher
Joined: Jun 26, 2008
Post Count: 2
Status: Offline
Reply to this Post  Reply with Quote 
smile Re: Minimum quorum reduced from 14 to 10 days for the Rice project

Is there a chance that in the future, the initial distribution will be reduce also? I crunch RICE exclusively for a while and notice that in most cases, after a result get validated, there're still 4 or 5 copies of that task still "In progress". smile
[Jul 30, 2008 6:54:25 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Minimum quorum reduced from 14 to 10 days for the Rice project

There's always looking at what the best minimum is without compromising scientific value. What i understand is that the results are monitored for sufficient structure, thus yes, there is chance. All depends on e.g. the average mix of computers that return results. It's a very very light project, so you really don't want to dump these over onto super duper computers i.e. leave enough also for the weaker shoulders to have their part of the Rice project.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Jul 30, 2008 7:00:33 PM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Minimum quorum reduced from 14 to 10 days for the Rice project

Is there a chance that in the future, the initial distribution will be reduce also? I crunch RICE exclusively for a while and notice that in most cases, after a result get validated, there're still 4 or 5 copies of that task still "In progress". smile


Please remember that every result returned for Rice is unique. With Rice we do not collect the results for the workunit until all results have reached a end state (late, error, success). Then ALL the valid results are sent back to the researchers for them to analyze (regardless if they were returned before validation was run the first time or not).

We do some statistics on the rate of results being returned and we expect that a majority portion of those results still 'in progress' even after validation will be returned so that we can collect and return to the scientists. They want about 120,000 predictions per gene. Each result contains between 50 and 500 structures depending on the difficulty of the gene and power of the computer. The way we do the analysis we are able to keep the structure generated pretty close to the desired target.

Setting the quorum to 10 simply means that you get your points quicker and it slightly changes our stats computation.
----------------------------------------
[Edit 1 times, last edit by knreed at Jul 30, 2008 7:16:15 PM]
[Jul 30, 2008 7:15:26 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Minimum quorum reduced from 14 to 10 days for the Rice project

knreed,

I'm somewhat confused... as I understand it, several copies of a Rice WU are sent out for computational analysis.

So let's say, 10 machines get it dispatched to them.

Now, each of these machines have differing computing capacity because their cpu speeds; memory speeds; quantity of memory; quantity of HDD work space are different.

Where I get lost in this is these WU's are rigged to run for approximately 8.00 hrs +/- about 0.10 units of time. So, if there 10 machines that are increasingly more powerful than the last, how does the most powerful computers results ever get completely validated?

Is it just a "well, all the other data from the machines below it match everything the fastest machine provided, hence, the data the fastest machine provided must be correct kind of syndrome?"

TIA
[Jul 30, 2008 7:53:36 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Minimum quorum reduced from 14 to 10 days for the Rice project

Barney, every copy is different. None of the work units do the same work.

Validation is done without comparing for an exact match.

Please read the project FAQ if you want to know more.
[Jul 30, 2008 8:18:59 PM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Minimum quorum reduced from 14 to 10 days for the Rice project

Barney,

Read through the info on these two pages (in particular the bottom section of the last page)

http://www.worldcommunitygrid.org/projects_showcase/rice/viewRiceMain.do
http://www.worldcommunitygrid.org/projects_showcase/rice/viewRiceAbout.do

The researchers take the structures produced and do some posting processing on the structures. Essentially, they are looking for clusters of predicted structures. Incorrectly computed structures show up and noise and are thus filtered out and therefore they can handle some error from the results.

We actually compute more than the 100,000 that they ask for to ensure that they have enough good results. We also added some error checking to the code so we can catch a large portion of the incorrect results.

--------
It is worth noting that a random seed is used by the application to initialize some starting parameters for each structure prediction. This is what makes each result unique.
----------------------------------------
[Edit 1 times, last edit by knreed at Jul 30, 2008 8:30:42 PM]
[Jul 30, 2008 8:28:46 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Minimum quorum reduced from 14 to 10 days for the Rice project

Barney, if you are interested in this topic, try reading about Monte Carlo simulation.
[Jul 30, 2008 8:38:22 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Minimum quorum reduced from 14 to 10 days for the Rice project

krneed & Didactylos ,

thanks I appreciate your responses.

Yes, I've jacked with Monte Carlo algorithms in the past.. been a few years but nevertheless, had to do it for one reason or another. Just wasn't completely sure how this was being done.

All I could see was that differing amounts of data per "WU" in a rice WU would likely be presented, presuming the initial dispatch would be the same.

But there's always a new twist.

Thanks again for the responses.
[Jul 30, 2008 8:51:11 PM]   Link   Report threatening or abusive post: please login first  Go to top 
[ Jump to Last Post ]
Post new Thread