Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 34
Posts: 34   Pages: 4   [ Previous Page | 1 2 3 4 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 3346 times and has 33 replies Next Thread
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: A 3 part Question on BOINC?

[Sending the 4th WU to a "reliable host" rather than a random host is an excellent idea. But going one step further... wouldn't it be nice if WUs could be flagged as either high priority or normal priority. BOINC and UD could spot the high priority flag and move that WU to the top of the host's work queue for that project or even to a position ahead of all projects if deadlines won't be missed.


This is an interesting idea - we will have to look at that. However, this change will be sometime in coming becuase it would require a change in the client.
[May 19, 2006 3:08:13 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: A 3 part Question on BOINC?

One comment here and the prioritisation. Depending on what i'm doing on my one man band machine, i sometimes postpone a WU, mostly FAAH, if i see ahead its going to take a long time....i prefer for these to run when i'm on one ear. On other occasions, sometimes i pull a few very short once forward (rosetta's usually, so they can make it before 24:00). Yes it's (my) lunacy, but the team wants every point they can put their hands on. This manual interrupt and release is not very elegant as you have to be at the machine to let one or the other go.

I actually thought a manual reorder would be nice, same as putting a priority code on it. Maybe thru a select, move up move down scroll.

Let me add another question (part 4): When the WU completes, in my case its send immediately (thus safely sotred on the WCG servers). It does not seem to enter the validation process until an actual secondary task report is send **. Some seem to take joy to save it up for days or a week, thus warping the stats. Now, does WCG take the WU into consideration for its internal purposes irrespective of receiving the second part i.e. stats running their own life? A comment here would be that if 3 are send only, WCG would we waiting a while before the quorum rule works..... worst case, up to 3 weeks.

** I thought to have seen the second part was a must for the validation, but cannot locate that comment.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[May 19, 2006 4:14:09 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: A 3 part Question on BOINC?

Yes it's (my) lunacy, but the team wants every point they can put their hands on. This manual interrupt and release is not very elegant as you have to be at the machine to let one or the other go.

I actually thought a manual reorder would be nice, same as putting a priority code on it. Maybe thru a select, move up move down scroll.



Buy a fishing rod. Maybe a small boat and motor too. You really need a diversion. Reordering WUs? My god man, what have they done to you?
[May 19, 2006 6:06:36 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: A 3 part Question on BOINC?

Happy as i am, happier they take away all those buttons to play with.....off to golf 06:30am Saturday Morning.... another lunacy (good time to start that long FAAH and grant it permanent 99% CPU time as i wont be back before dusk wink
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[May 19, 2006 6:41:05 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: A 3 part Question on BOINC?

[Sending the 4th WU to a "reliable host" rather than a random host is an excellent idea. But going one step further... wouldn't it be nice if WUs could be flagged as either high priority or normal priority. BOINC and UD could spot the high priority flag and move that WU to the top of the host's work queue for that project or even to a position ahead of all projects if deadlines won't be missed.


This is an interesting idea - we will have to look at that. However, this change will be sometime in coming becuase it would require a change in the client.


I was thinking that the decision to stick with the "3 WUs now, a 4th later if required" practice may ultimately depend on how quickly results for the 4th WU can be returned. Having a high priority flag could ensure 4th WUs get crunched and returned with minimal turn around time. It's just an augmentation to sending 4th WUs to reliable hosts.

My idea was that the server would raise the high priority flag and that the flag would be buried and unavailable for overly zealous host operators to fiddle with in some insane effort to maximize their team's points (sheesh!)

Yes, I understand such an idea would be "sometime in coming", no problem here with that.
[May 21, 2006 12:07:41 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Johnny Cool
Ace Cruncher
USA
Joined: Jul 28, 2005
Post Count: 8621
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: A 3 part Question on BOINC?

Workunits are now being distributed that only send out three results at the start. We will be evaluating the effectiveness over the next couple of weeks. I will let everyone know what the new send/recieve ratio per workunit is once we have sufficient data.

From my post above, the current average ratio is that for each workunit 4.2 results are sent and 3.6 results are received.


Why was there not a post in "Known Issues" or especially here in the "Boinc Agent Support" thread about the delays with Boinc validations?

I actually had to read a few Team threads to learn about this issue? thinking

I have over 28 hours of crunching (like many of you) going back at least two days that have not "recorded" results and what not.

I think that I have only been credited with one result this evening.

Edit: Yes, only one result.
----------------------------------------

Team Andrax Co-Captain
Free-DC Stats
Join Team Andrax at WCG
----------------------------------------
[Edit 3 times, last edit by Johnny Cool at May 23, 2006 1:10:02 AM]
[May 23, 2006 1:03:40 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: A 3 part Question on BOINC?

[Sending the 4th WU to a "reliable host" rather than a random host is an excellent idea. But going one step further... wouldn't it be nice if WUs could be flagged as either high priority or normal priority. BOINC and UD could spot the high priority flag and move that WU to the top of the host's work queue for that project or even to a position ahead of all projects if deadlines won't be missed.


This is an interesting idea - we will have to look at that. However, this change will be sometime in coming becuase it would require a change in the client.


Speaking from the vantage of someone running a BOINC client attached to ten different projects, ... I have to say that I'd find the idea, if carried as far as permitting a project to signal that an assigned WU gets priority over other projects' WUs, is objectionable on two points:

Firstly, when I set up my own resource allocation, for the most part I give more resources to those projects that matter more to me. They all matter to me, but some are - from my own bias - worth more of my attention. Create a system which overrides the participants' judgement and I don't think that the BOINC community as a whole would respond positively.

Secondly, and probably the stronger point, the acquisition of funding and resources in world of academic and private research is already extremely competitive and - at its worst - quite vicious. Partly that's created by the scarcity of resources. Partly that's because the "small world" phenom. is perceptibly more pronounced in these circles. And several other reasons as well, but I'm not going to write a dissertaion on the sociology of science right here and now. Suffice it to say that if you piss someone off, and you'll likely regret it, as you will almost certainly meet up again at a time when they are in a position to make you uncomfortable.

Creating a system that potentially gives one project the ability to trump another in BOINC resource allocation is just begging for trouble. And I'd be a bit offended if my computer was essentially part of a battleground in a needless feud between projects that are both worthy of resources. It might not happen if a priority system were implemented, but I'd rather the temptation not even be there.

And, also, some of the worst science is borne of these kinds of feuds. Paleontology is, last I heard, still sorting out some of the misinformation created out of a rather bitter feud between two paleontologists which had turned into a contest to see who could name the most dinosaurs. I don't think the same kind of dramatically bad science would play out here, but the last thing anyone need do is give any feud any more amunition with which to fight.

(Disclaimer: All the above said might give you the impression that I think most scientists are childishly argumentative and petty. Not the case. I think that nearly as many scientists are childishly argumentative and petty proportionate to the population of us as a whole. But it's always astounding to see just what degree of damage a very small minority of particularly selfish and vindictive individuals can achieve. The only thing that keeps this fact from overwhelming the rest of us is that vast amounts of good can be done by very small minorities of particularly generous and empathetic people.)
[May 27, 2006 8:51:28 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: A 3 part Question on BOINC?

Don't worry, any such prioritisation would have to be implemented on the client, and as such all BOINC projects would benefit.

I don't think it's particularly likely that any such system would ever be implemented, but supposing it were.... it would probably work merely by reordering work units. The BOINC client already has code to respect the project weightings and avoid missing any work unit deadlines. It is paramount that this should continue to be the case.

Going to the absolute extreme, and allowing work units to be bumped - well, that would have to be configurable. I can see a lot of objections to it, but there is an up side, too.
[May 27, 2006 9:29:21 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: A 3 part Question on BOINC?

As a postscript to what I wrote earlier - I realize I didn't write this in as I'd meant to: if a single project wants to be able to prioritize their own WUs on the fly, I see no reason why not. It would be a nifty feature. And, as you pointed out, preemption would still manage the overall allocation. (I'm a bit confused, though, on how preemption still applies if an inter-project priority system is implemented.)

I also do kind of wonder ultimately about the overall value for the effort, regardless of the geeky-coolness factor of having a priority-queue system of sorts -- why not just (as WCG has started to realize the possibility of) send priority WUs out to those machines which are fastest and most likely to return them reliably? It would accomplish much the same thing, although it wouldn't be able to get around projects with monolithic WUs (a la CPDN).

Bottom line, though, you're almost certainly right - to a degree it even takes on a tone of understatement - that it'd probably be some time before such a feature were incorporated in the BOINC client.

Speaking of CPDN, one other reason I really would not want to see one project trumping another is because of what I saw after taking on a BBC/CPDN work-unit. After about two weeks I checked the percent complete and found that it had no hope of completing before the "generous" one-year deadline. I'm not certain that the method my BOINC-client is using to figure out when it is over-comitted works for me, nor the method used for recovery. The BBC/CPDN work-unit is the only one ever in jeapordy of not being completed, and of course the one time the client detected it had accepted too much work that WU was at the end of the queue because of the first-due-first-finished reprioritization. It is only by throwing the monsterous 2000+ hour WU nearly half of the resources that I've found a way to ensure it has a chance to complete. Something else being permitted to jump ahead in line, depending on its size, could potentially put the client back in "panic" mode and potentially always stay ahead.

I'm not sure that creating a system which generates 2000+ WUs doesn't constitute some kind of arm-twisting for those of us who are attached to multiple projects and would like to allocate things at least a bit more evenly. But leaving that project switched to "no new work" for a duration after that one WU is complete should eventually balance things out. :-> If that project is still going once my efforts have balanced out closer to where I'd like to see them, then I'll take more work from them then.
[May 28, 2006 1:21:43 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Johnny Cool
Ace Cruncher
USA
Joined: Jul 28, 2005
Post Count: 8621
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: A 3 part Question on BOINC?

However, this will impact some users because the fourth result did play a role in the grid. About 15% of the results that we send out are not returned with a successful result. Most of these are due to the result never being returned. BOINC will automatically handle this condition by detecting that a result has not been returned and it will send the workunit out to an additional computer. However, this will impact users because in the event that one of the results is not returned they will have to wait until the workunit is sent out again and the result is returned before their results can be validated. This means that in some cases members will experience an additional delay in being awarded credit for the work they have returned.

I guess that I am included in "some cases".

After crunching for 72 hours straight, I have this to report for today:

05/27/2006 0:000:05:48:51 414 Points 2 Results Returned

Less than six hours? thinking

This is progress? 414 Points and 2 Results Returned?
----------------------------------------

Team Andrax Co-Captain
Free-DC Stats
Join Team Andrax at WCG
[May 28, 2006 1:27:07 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 34   Pages: 4   [ Previous Page | 1 2 3 4 | Next Page ]
[ Jump to Last Post ]
Post new Thread