Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 18
Posts: 18   Pages: 2   [ Previous Page | 1 2 ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 2063 times and has 17 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: New Status in Column?


Plus once when a high priority job starts and makes this task "Waiting to run",
plus once when another completing task allows this waiting task to resume,
plus once when you decide to suspend one task, and once when you resume it,
plus when you decide to suspend network activity (but how will it do it?)
plus when you switch to another Grid or when you snooze your PC or when you stop it completely or when you leave for vacation, or.....


Jean,

Thanks for that... however if you noticed, I didn't suggest all those other things... I'll admit I thought of it and decided it would become unwieldy.

I'm sure we both understand there are only two techniques to gather performance information. Statistical sampling and specific events.

Statistical sampling I'm fairly sure is being performed on the client.

Attempting to have the server perform statistical sampling of the clients would require to much resource from the server and the client both to have it done successfully. I'm not saying it couldn't be implemented, but the question is, how specific does it need to be?

This is exactly why I suggested only reporting when the WU was selected for computational processing on the client should it be reported. The rest of the events should be ignored.

The win-win is that it presents information back as to when to actually start expecting the data and if it has any opportunity to have the results for the WU being returned in time.

So I did actually give all the other points some consideration for measurement and reporting; but it's just not realistic for this kind of load on the network.

Just having the data that the WU has started should be sufficient. A much better status than "In Progress" and not knowing where in client's queue the WU is / might be / and how long it takes that client (on average) to complete the WU on that system.

That's just my thinking, I'm a simple kind of guy and figured it's implementation would have a fair amount of benefit for everyone involved. While not precise, much better than what's available today.
----------------------------------------
[Edit 1 times, last edit by Former Member at Jul 28, 2008 7:12:29 AM]
[Jul 28, 2008 7:01:37 AM]   Link   Report threatening or abusive post: please login first  Go to top 
JmBoullier
Former Community Advisor
Normandy - France
Joined: Jan 26, 2007
Post Count: 3716
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: New Status in Column?

Barney,
First, please allow me some friendly teasing: I know you did not ask for all these things to be taken into consideration. I was only trying to anticipate one or two or your future posts... smile And I can risk one more guess: if all this is accurately reported in the Status page it will become obvious that we will dramatically miss an estimate of the time the other WUs will have to wait in the queue before starting, no? smile

OK, seriously now.
You suggested a new WU status in the Status page, i.e. a new information for human eyes. The servers do not read the Status page, actually they are providing info for it. If your concern is in fact that the servers should have interesting information about what is currently running in your PCs, then it's a different story, and actually they do have such information.

Go to your Boinc directory, make a copy of the file called "sched_request_www.worldcommunitygrid.org.xml" to not disturb your Boinc client, and then have a look at its contents. Look for the section called "in_progress_results" at the end of this file and you will see that the servers know what is going on in your PC every time your Boinc client asks for more work.

For human eyes, as Lawrence said, BoincMgr or BoincView are much more convenient for monitoring the life of your client. And the fact that you cannot know about what is going on inside the clients of other members with the same WUs is not only a matter of keeping the Results page not excessively expensive to generate, it may also be for protecting the privacy of other members, a concern which is taken seriously by WCG/IBM.

Cheers. Jean.
----------------------------------------
Team--> Decrypthon -->Statistics/Join -->Thread
[Jul 28, 2008 8:19:09 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: New Status in Column?

Jean,

Sure friendly teasing is a nice break and it's fun to banter.

I'm not exactly sure what you are trying to suggest with this:

And I can risk one more guess: if all this is accurately reported in the Status page it will become obvious that we will dramatically miss an estimate of the time the other WUs will have to wait in the queue before starting, no? blushing


Let's see if by way of example this might help a little bit and use the following screen shot.



Here is what I know.

1). This WU is working on my system as I type this. It should take about 3.2 hrs to complete.
2). This WU was dispatched to two at about the same time.
3). using the above screen no one can tell the accurate status of the tasks because "In Progress" is used synonymously when the WU is awaiting to be scheduled on the queue; running and I'll presume for a period of time when it's trying to report.

I have WU's as results that have not been validated all the way back to 7/17 with final completion dates of 6, Aug. When I select the detail view, all I can see is In Progress, error, invalid and Awaiting Validation. (see screen shot below).



So for those of us who are wondering the status, when viewing the above page would at least let someone know it's running.

As for the privacy issue, yes I understand those concerns. Notice however, there is nothing that identify s anyones machine other than your own machine. In my case, it's the orange bar on the report (not in this one, but is in the one above).

As for protecting the privacy of other members, please make note that the information in the above screen shots does not reveal anything about whom the other participants assigned to work on this WU. What it does provide is information about the general performance of another machine in comparison (loosely speaking of course), but absolutely nothing about whom nor where.

BTW, I do appreciate eveyones observation here but I would (and I suspect others might as well) appreciate seeing this as a new status in the field of the above display.

I do use bionic to view stuff... but there are some things that one needs to go back to the WCG web pages to acquire.

While I was preparing a reply, my system got another DDDT to run.. and looking at the detail from the WCG status page one can see the following:



Meaining, the WU was sent out on 23, July to two machines; one responded; one didn't so it was sent out as either an emergency or repair on 27, July with no reply and then it was subsequently re-dispatched as an emergency or repair on 27, July and I can tell you that baring any power disruptions the results will be provided within the next 90 minutes or so.

So having a "running" status on this page seems more and more reasonable to me so the rest of us can have a little better glimpse into what's going on.

I know, I'm the one at age 2 that didn't care about toys, I wanted to understand what made the clock tick because I could hear it biggrin Now that I'm a tad older, now I want to see the speedometer (and this is kind of a speedometer to me).... I wanna see when the train is going to reach its destination!
----------------------------------------
[Edit 1 times, last edit by Former Member at Jul 28, 2008 2:19:14 PM]
[Jul 28, 2008 2:05:10 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: New Status in Column?

Hi, Barney. As others have tried to say, this is a technical issue, not a philosophical one. It simply isn't practical to track the running state of every single work unit on the servers. It would grind the servers to a halt!

All the server knows is the status of your client at the time it requests new work. Updating the status every time the client state changes would require dozens more server contacts.
[Jul 28, 2008 2:11:00 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: New Status in Column?

Think all the points have been made why you wont be seeing in the status column here or any other project if a job is actually being processed. You might want to give the idea a go at the Berkeley developers forum. It's a too major increment in communications load and if implemented for the smaller projects a major additional strain on their server system. Certainly it would require a database expansion,which does not only impact WCG, but every BOINC utilizing DC project.

BOINCview does a great job of tracking your own jobs on as many clients you connect to, simultaneous. It only needs one windows host to run it from. It tallies to include all completed jobs beyond the date where all BOINC projects remove the work units after passing through the assimilators.

We live with the deadlines set, 12 days for all but HPF2 which has 20 days. We know well over 90% of the work validates within 4 days, so we exercise the patience. I've got a job sitting in PV since July 13. Does not bother me. Periodically I filter on error and invalid and see what happened with them.

Only mod that has been proposed is being able to see the result logs of those in the quorum, so we / the members can check and compare if their own results are anomalous, but there is hesitation as with that tool there will be those that start cancelling jobs without the expertise to properly judge .... now I've strayed.

Paramount is, that the servers can work and result data can flow in the most efficient manner. Overloads already resulted in WCG increasing job sizes several times to stop situations where folk were not getting work.

Just look for some posts by knreed why some things are not there simply because database hits are costly to the system. A very major concern, which i am not going to argue over how it could be done better or more efficient. That is for discussion at the Berkeley developers forum, where many an expert fields answers who actually themselves developed and run projects.

ciao
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Jul 28, 2008 2:29:16 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: New Status in Column?

Hi Didactylos,

Has anyone done any modeling on this at all to see what the real effects would be?
[Jul 28, 2008 2:29:55 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: New Status in Column?

Instead of being proportional to the number of active members, server load would be proportional to the number of results + the number of active members.

The precise impact would depend on the project - the size of the work units, and the average host efficiency. Off the top of my head, this means that in the best case, it would triple server load.

WCG (and other projects) are already severely constrained by the scheduling server. Putting significantly more load on it for no substantive gain really isn't an option.

Sorry, Barney. Nice idea though it is, it's a non-starter. Further analysis would be a waste of time - there are more productive areas to work on (and there's a lot of work to be done!)
[Jul 28, 2008 2:44:08 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: New Status in Column?

Hi BarneyBadass,

There are a number of tightly coupled computer clusters (grids) that can easily do what you want. These are mostly on university campuses and are maintained and developed by the computer science department as a research project. If you read their web pages, you will be amazed by the number of problems affecting their grids that they have been working on for more than a decade. We are trying to avoid headaches by keeping it simple.

We also have to be sensitive to privacy issues that are not very important on a university campus. BOINCView does all that you want on a local LAN. Before we tried to make BOINC act the same way, we would have to get privacy permissions from every member whose information we put out in public. Sound silly? There are no generally agreed policies on private and public information. This will slowly be worked out this coming century, but we are avoiding the whole problem by showing what is on our servers rather than what is on our member's computers.

Lawrence
[Jul 28, 2008 3:56:05 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 18   Pages: 2   [ Previous Page | 1 2 ]
[ Jump to Last Post ]
Post new Thread