Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 20
Posts: 20   Pages: 2   [ Previous Page | 1 2 ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 1695 times and has 19 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Impact of double validation on performance

Sek's *hint* this time is that WCG is very stable. It does not crash and is only occasionally taken down for maintenance and we get advanced notice when that is going to happen. WCG runs spectacularly when compared to just about any other project (gratz to the smart people behind the scenes).
The longest downtime I have seen recently (probably in th last year) is 6 hours ... I have a cache of .35 (8 hours) to make sure I don;t run dry. In some small way I think it helps WCG in general when I turn makeup WUs around quickly.
[Dec 9, 2009 1:10:29 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Hypernova
Master Cruncher
Audaces Fortuna Juvat ! Vaud - Switzerland
Joined: Dec 16, 2008
Post Count: 1908
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Impact of double validation on performance

Oh! with all this crunching I forgot.

Merry christmas to all of you and my best wishes for you to continue crunching in good health with all your family.
----------------------------------------

[Dec 9, 2009 1:27:11 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Impact of double validation on performance

Hypernova,

Look at the My Grid, My Statistics > My Stats History, to see the records of your daily validations. That's the one to use. One line lower you can even get it per active device

[Very OT]Have a GTX220, 1gb DDR2 waiting very patiently... CUDA drivers 195.24 WHQL in place, OpenCL ready to run. Snapped up last week for a very snappy price. It says 16 cores on the box, whatever that means, PCI x16 slot user and that's what is being used now by an old GS8300. [/Very OT]
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
----------------------------------------
[Edit 1 times, last edit by Sekerob at Dec 9, 2009 1:32:32 PM]
[Dec 9, 2009 1:30:58 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Hypernova
Master Cruncher
Audaces Fortuna Juvat ! Vaud - Switzerland
Joined: Dec 16, 2008
Post Count: 1908
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
smile Re: Impact of double validation on performance

Sekerob,

you are absolutely right, and it makes me feel much better.

Over my famous 9 day period I returned the very nice number of ........................................557 WU and not the 264 as mentioned before.

So my validation rate has increased and stands now at :

90% Yeeessssssss peace
----------------------------------------

[Dec 9, 2009 3:26:07 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Ingleside
Veteran Cruncher
Norway
Joined: Nov 19, 2005
Post Count: 974
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Impact of double validation on performance

If my reasoning is right then there is no incentive to have very powerful machines crunching. But I suppose I am wrong somewhere.

As long as your "very powerful machine" doesn't manage to crunch atleast 50% of all WCG-wu's sent-out each day devilish , you'll reach a "steady-state" there you'll on average crunch X wu/day, and also on average X wu/day will be validated...

Instead of having a single machine, more realistically you'll have a large farm of computers crunching under your name. But, as long as you're not responsible for 50% ** or more of total production, there isn't really any problem. It can take a few days from you starts running before you've reached a steady-state, but after you do, you'll get more or less a continous stream of granted credit. There will of course be some daily variations, but if you instead of daily looks on weekly production, there will be less variations, and basically the weekly granted will be directly linked to how much you've crunched.


** Well, in sub-projects that uses Homogeneous Redundancy, if you're using a fairly uncommon cpu-type (or OS), it's possible there aren't really anyone else running it, so you'll either completely block the work-queue in this sub-project (you grabs 50% of the work, and no-one else can get any work since wrong cpu/OS-combination), or project must re-configure so your computers validates eachothers results.

So, in practice you'll likely get problems if your computers is nearing 50% of total production in a sub-project, but no-one is even close to reaching this kind of production in WCG.
----------------------------------------


"I make so many mistakes. But then just think of all the mistakes I don't make, although I might."
----------------------------------------
[Edit 1 times, last edit by Ingleside at Dec 9, 2009 8:44:03 PM]
[Dec 9, 2009 8:43:19 PM]   Link   Report threatening or abusive post: please login first  Go to top 
gb009761
Master Cruncher
Scotland
Joined: Apr 6, 2005
Post Count: 3010
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Impact of double validation on performance

So, in practice you'll likely get problems if your computers is nearing 50% of total production in a sub-project, but no-one is even close to reaching this kind of production in WCG.


Not even Mr. Kermit when he had everything powered up for that short time biggrin
----------------------------------------

[Dec 9, 2009 9:31:43 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Impact of double validation on performance

This 50% story is far divorced from reality. Not only that, distribution of quorums is allowed to be to multiple devices of the same member i.e. validation flow would not be impaired. Lastly, homogeneity down to rare CPU's is still rare and might, but improbable and very remotely play only a role for HCMD2. WCG in past has taken such distributions out from eligibility.

edit: Oh yes, 1 machine, monster... would not get more than currently 300 or so tasks per day. Each of the projects has ~60,000 or more results daily validating. This monster manager needs to negotiate with the techs.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
----------------------------------------
[Edit 2 times, last edit by Sekerob at Dec 9, 2009 10:47:41 PM]
[Dec 9, 2009 10:39:15 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Impact of double validation on performance

Sekerob,

you are absolutely right, and it makes me feel much better.

Over my famous 9 day period I returned the very nice number of ........................................557 WU and not the 264 as mentioned before.

So my validation rate has increased and stands now at :

90% Yeeessssssss peace

Not clear if you did, but was the 557 the validated count from the stats hist i.e. exclusive of the PV jobs? If not, then do 557 / (54 +557) = 91.1%. biggrin How many of the 54 PV were older than 4 days? < 10% it should be.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
----------------------------------------
[Edit 1 times, last edit by Sekerob at Dec 10, 2009 7:05:01 AM]
[Dec 10, 2009 7:00:48 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Hypernova
Master Cruncher
Audaces Fortuna Juvat ! Vaud - Switzerland
Joined: Dec 16, 2008
Post Count: 1908
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
smile Re: Impact of double validation on performance

Sekerob,

I tried now the measure things on a 7 day period which is what My Stats allows.

So over the last 7 days I get with My Statistics 498 WU returned and points counted. I understand these are Validated WU's.

Of these 349 were returned over the last 4 days. So you have 70% over the last 4 days.

If I look at My Device Statistics over the same 4 days period I get 805 WU's returned. But here I suppose it is everything and not only validated one's. Are the In progress WU's also in this total or not?

But I must say I still feel there are incoherences in all these numbers.
That's not too important anyway.

What counts is that I am at 1.84M points and my target is 2Million by the 17th of December which will make exactly 1 year I started crunching.

At the beginning it was really little stuff but I am improving. I started with 4 physical cores and no HT and running part time. Now I am at 16 Physical cores and 32 virtual all i7 and all over 3GHz. Of these 16 will run fulltime WCG only so I should reach my year end target. This will be my christmas gift.
----------------------------------------

[Dec 10, 2009 9:10:44 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Impact of double validation on performance

Device Statistics/History is just a subdivision of the total stats i.e. only that what is valid. If you drill into the active devices and copy the daily values, add them up it should tie back to the overall daily stats. Then if you expand the device filter to show all, in case you have old installs, the sum of those should add back to your My Grid total. The Result Status page is real time whilst the Device/My Grid stats only update twice a day, so those 2 will very rarely tally.

There's a small but, the overall stats including spill. The servers keep tallying and validating during the statistics compilation. So right now I see in my totals 1 result that validated at 00:05:01 UTC on Dec.10. The WCG overall stats presently include a total spill of 1 year 127 days i.e. 1777 results are included for the cumulative total of WCG as at the 9th, but belonging to the 10th.

12/10/09 1:127:06:49:52 1,139,953 1,777
12/9/09 305:033:20:11:13 259,806,378 407,433

Yes, WCG accounts results better than a bank.

Confusing it may be and not really important... the point is, that if you returned a result without a outright error code they will eventually add into your contribution totals, without a shadow of a doubt. Even invalid results add into your stats, but have not seen one myself for a very long time.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Dec 10, 2009 9:35:15 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 20   Pages: 2   [ Previous Page | 1 2 ]
[ Jump to Last Post ]
Post new Thread