Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 80
Posts: 80   Pages: 8   [ Previous Page | 1 2 3 4 5 6 7 8 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 9564 times and has 79 replies Next Thread
JmBoullier
Former Community Advisor
Normandy - France
Joined: Jan 26, 2007
Post Count: 3716
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: BOINC: Roadmap (we want your input!)

Did not mean a contention... there is none. When the line is open with the client, it attempts to squeeze as much in, in terms of transfer.... uploads usually much slower than downloads (ISP restriction here). It may thus have intend to ask just before finishing. It just comes to mind that there are these fancy features that allow to tell the client NOT to start a job if it turns out to be redundant or junk.... adherence to security: Client initiated contact.

I speculate.
Right Sek, you speculate and you complicate. smile
What I am addressing is a pure timing problem inside BOINC.
When I started this topic I thought I would no longer have any trace of the case. Fortunately stdoutdae.txt is larger than I thought and I have found two different cases which demonstrate it. Watch the bold lines and the times.

2007-06-27 15:13:25 [World Community Grid] [checkpoint_debug] result 15000544-15001426_2 checkpointed
2007-06-27 15:20:21 [World Community Grid] Computation for task 15000544-15001426_2 finished
2007-06-27 15:20:21 [World Community Grid] Starting 15000694-15001426_1
2007-06-27 15:20:22 [World Community Grid] Starting task 15000694-15001426_1 using fcg1 version 514
2007-06-27 15:20:22 [World Community Grid] Sending scheduler request: To fetch work
2007-06-27 15:20:22 [World Community Grid] Requesting 3722 seconds of new work
2007-06-27 15:20:23 [World Community Grid] [file_xfer] Started upload of file 15000544-15001426_2_0
2007-06-27 15:20:27 [World Community Grid] Scheduler RPC succeeded [server version 509]
2007-06-27 15:20:27 [World Community Grid] Deferring communication for 5 min 3 sec
2007-06-27 15:20:27 [World Community Grid] Reason: requested by project
2007-06-27 15:20:30 [World Community Grid] [file_xfer] Finished upload of file 15000544-15001426_2_0
2007-06-27 15:20:30 [World Community Grid] [file_xfer] Throughput 11716 bytes/sec
2007-06-27 15:20:30 [World Community Grid] [file_xfer] Started download of file fcg1.15000228.faa
2007-06-27 15:20:30 [World Community Grid] [file_xfer] Started download of file fcg1.15001437.faa
2007-06-27 15:20:45 [World Community Grid] [file_xfer] Finished download of file fcg1.15000228.faa
2007-06-27 15:20:45 [World Community Grid] [file_xfer] Throughput 30378 bytes/sec
2007-06-27 15:20:45 [World Community Grid] [file_xfer] Finished download of file fcg1.15001437.faa
2007-06-27 15:20:45 [World Community Grid] [file_xfer] Throughput 30134 bytes/sec
2007-06-27 15:30:27 [World Community Grid] [checkpoint_debug] result 15000694-15001426_1 checkpointed
Had BOINC delayed the "request to fetch work" until after the "finished upload" one the just-finished WU would already been "Pending validation" on the server side instead of staying "In progress" for another while.
Another case slightly different but with the same result.

2007-06-27 18:15:02 [World Community Grid] [checkpoint_debug] result 15000821-15001423_0 checkpointed
2007-06-27 18:21:41 [World Community Grid] Computation for task 15000821-15001423_0 finished
2007-06-27 18:21:41 [World Community Grid] Starting 15000925-15001427_2
2007-06-27 18:21:42 [World Community Grid] Starting task 15000925-15001427_2 using fcg1 version 514
2007-06-27 18:21:44 [World Community Grid] [file_xfer] Started upload of file 15000821-15001423_0_0
2007-06-27 18:21:44 [World Community Grid] Sending scheduler request: To fetch work
2007-06-27 18:21:44 [World Community Grid] Requesting 1717 seconds of new work
2007-06-27 18:21:49 [World Community Grid] Scheduler RPC succeeded [server version 509]
2007-06-27 18:21:49 [World Community Grid] Deferring communication for 5 min 3 sec
2007-06-27 18:21:49 [World Community Grid] Reason: requested by project
2007-06-27 18:21:51 [World Community Grid] [file_xfer] Finished upload of file 15000821-15001423_0_0
2007-06-27 18:21:51 [World Community Grid] [file_xfer] Throughput 11712 bytes/sec
2007-06-27 18:21:51 [World Community Grid] [file_xfer] Started download of file fcg1.15000539.faa
2007-06-27 18:21:59 [World Community Grid] [file_xfer] Finished download of file fcg1.15000539.faa
2007-06-27 18:21:59 [World Community Grid] [file_xfer] Throughput 50292 bytes/sec
2007-06-27 18:31:45 [World Community Grid] [checkpoint_debug] result 15000925-15001427_2 checkpointed
Without forgetting the many cases I can show like this last one:
2007-06-26 06:26:42 [World Community Grid] [checkpoint_debug] result 15000570-15001408_2 checkpointed
2007-06-26 06:28:34 [World Community Grid] Sending scheduler request: To report completed tasks
2007-06-26 06:28:34 [World Community Grid] Reporting 1 tasks
2007-06-26 06:28:39 [World Community Grid] Scheduler RPC succeeded [server version 509]
2007-06-26 06:28:39 [World Community Grid] Deferring communication for 5 min 3 sec
2007-06-26 06:28:39 [World Community Grid] Reason: requested by project
2007-06-26 06:30:26 [World Community Grid] Computation for task 15000570-15001408_2 finished
2007-06-26 06:30:26 [World Community Grid] Starting 15000731-15001414_0
Watch the times. By waiting 3 minutes it could have reported two tasks instead of one.

Do you better see what I mean now? A little more communication between the different routines of BOINC could improve performance on the server side by much, particularly for projects which have shorter WUs.

Cheers. Jean.
----------------------------------------
Team--> Decrypthon -->Statistics/Join -->Thread
[Jul 7, 2007 8:57:14 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: BOINC: Roadmap (we want your input!)

I understood the first time - I should have replied then, before Sek jumped in ;-)

Yes, this is going to happen sometimes. And sometimes a work unit is going to upload and then be reported seconds later. Statistically, either is just as likely. On average, an upload will be timed just as close to the previous scheduler request as to the next request.

I suppose that the work unit time could rarely cycle with the report time, so that uploads frequently miss the report. But this is the worst case, and is an acceptable scenario.

So, the question is: is it worth trying to bias the requests such that they tend to fall just after uploads? I don't think so. If there is an urgent reason for reporting the result, then a special scheduler request will be made. Otherwise, it can wait.

However, the situation you are describing is a good argument for not setting the "connect to network every" setting too high. The WCG default is 0.1 days, or 2.4 hours. With BOINC 5.10, users will be able to leave that setting small, and use the additional work buffer to maintain their queue.
[Jul 7, 2007 9:11:09 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: BOINC: Roadmap (we want your input!)

Do you better see what I mean now? A little more communication between the different routines of BOINC could improve performance on the server side by much, particularly for projects which have shorter WUs.

Cheers. Jean.

Yes Jean, i see the same as what you described before and it's identical in the 5.10.8 logs of a machine with 1.0 days connect, 3 days additional buffer and network always on. Set additional connect from e.g. 23:00 to 24:00 UTC ** and "network activity based on preferences" and you'll find the whole up/down condensed to that hour. All get the points.... I'm sure i wrote this same scheme yesterday too.

Have a great day.

** (Connect to the Internet only between: 23:00 and 24:00 ) can be done from 5.10 on web profile or in client directly but refined to minutes.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Jul 7, 2007 10:18:40 AM]   Link   Report threatening or abusive post: please login first  Go to top 
twilyth
Master Cruncher
US
Joined: Mar 30, 2007
Post Count: 2130
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: BOINC: Roadmap (we want your input!)

Considering the fact that WCG is just one implementation of boinc - as compared to the numerous other projects (and soon commercial endeavors such as gridfinity) where it is used - is it realistic to believe that our input will have any significant impact on future development?

I would guess that the suggestions that will be taken most seriously will be those that come from projects with a much larger user base than WCG (seti for example). If some of ours happen to mirror what comes from those projects, they will be considered with or without our input. And if not, can we expect the boinc developers to give our suggestions similar weight?
----------------------------------------


[Jul 7, 2007 1:23:21 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: BOINC: Roadmap (we want your input!)

is it realistic to believe that our input will have any significant impact on future development?

You bet! Good suggestions will be implemented, if at all possible. You can count on it.

WCG already added the Simple View. And remember - the majority of BOINC projects are absolutely tiny. WCG are one of the big 5, and if we move over entirely to BOINC, WCG will be head to head with SETI.

It doesn't matter who the idea comes from - it only matters that it's a good idea.
[Jul 7, 2007 1:33:36 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: BOINC: Roadmap (we want your input!)

WCG is actually top 3 on daily production and with the migration of UD to BOINC agent only getting stronger in global participation compared to other projects and see no surprise if the second spot is taken in not too distant time. See Here:

http://boincstats.com/index.php?or=21

There is actually quite a rich 'behind the scene' exchange on how too's and shared to the open source platform and observed several credits given to WCG at other projects for improving server / client management.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Jul 7, 2007 2:15:04 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: BOINC: Roadmap (we want your input!)

Thank you, everyone, for your ideas so far. I'm going to begin collating them soon.

Meanwhile, here are some topic areas that might spark some more ideas:

The installation experience
The simple view
The graphics screen
The advanced view
The preferences system

People with many computers, are there any special features that you would like?
[Jul 9, 2007 3:28:02 PM]   Link   Report threatening or abusive post: please login first  Go to top 
JmBoullier
Former Community Advisor
Normandy - France
Joined: Jan 26, 2007
Post Count: 3716
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: BOINC: Roadmap (we want your input!)

Didactylos,
I wonder, wouldn't it be more clear and simple if you changed the title of this thread to something like "BOINC future releases: we want your input!". For people who have never used the BOINC forum it might not be clear what "roadmap" actually covers. It's being used a little for anything and the rest nowadays in the political world where it does not always make people think "practical". smile

Cheers. Jean.
----------------------------------------
Team--> Decrypthon -->Statistics/Join -->Thread
[Jul 10, 2007 1:13:13 AM]   Link   Report threatening or abusive post: please login first  Go to top 
JmBoullier
Former Community Advisor
Normandy - France
Joined: Jan 26, 2007
Post Count: 3716
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: BOINC: Roadmap (we want your input!)

The timestamping in local time of BOINC's messages is OK for the end user and that should not change. However when it comes to problem solving it would help a lot if the UTC (GMT) offset is added, in particular when problems involve the server side.

Currently it is impossible when seeing an extract of the message log in a post to figure out when in the day it has actually happened.

This offset is not even in the initial messages nowadays.

So I suggest something like "10/07/2007 03:15:43 UTC+2" as the new timestamp. But maybe the string UTC itself is superfluous?

Cheers. Jean.
----------------------------------------
Team--> Decrypthon -->Statistics/Join -->Thread
[Jul 10, 2007 1:36:34 AM]   Link   Report threatening or abusive post: please login first  Go to top 
retsof
Former Community Advisor
USA
Joined: Jul 31, 2005
Post Count: 6824
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: BOINC: Roadmap (we want your input!)

So I suggest something like "10/07/2007 03:15:43 UTC+2" as the new timestamp. But maybe the string UTC itself is superfluous?
There is a letter for each of 24 basic time zones. The only one in common use over here is Z/Zulu for UTC or 0000Z. There are several more fractional time zones, but that is another issue.

http://en.wikipedia.org/wiki/List_of_time_zones

Pakistan is UTC+5 E, so naturally India is UTC+5:30 E*. If India is UTC+5:30, Nepal has to be UTC+5:45 E**.

etc. etc.

Islands on either side of the International Date Line can skip or remove whole days now and then.

What time is it at the South Pole? New Zealand time, since that's where most flights come from.
----------------------------------------
SUPPORT ADVISOR
Work+GPU i7 8700 12threads
School i7 4770 8threads
Default+GPU Ryzen 7 3700X 16threads
Ryzen 7 3800X 16 threads
Ryzen 9 3900X 24threads
Home i7 3540M 4threads50%
[Jul 10, 2007 2:44:17 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 80   Pages: 8   [ Previous Page | 1 2 3 4 5 6 7 8 | Next Page ]
[ Jump to Last Post ]
Post new Thread