| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Completed Research
Forum: The Clean Energy Project - Phase 2 Forum
Thread: [Help needed] Can't receive CEP Phase 2 tasks due to network bandwidth issue |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 54
|
|
| Author |
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
The techs are aware of such borderline kick outs since the 2nd which seem to appear mostly when individual clients have frequent upload issue.
----------------------------------------Fri 02 Jul 2010 17:48:27 CEST WCG Message from server: The Clean Energy Project - Phase 2 requires 128.00 kbps download bandwidth. Your computer has been measured at 126.77 kbps. Patience please... CEP2 will run for quite some time :D Down here in the MED outback, still on a CEP2/HPF2/HCMD2 diet ad interim (since those researches still want to be turned blue anyhow) and getting a ratio of about 74% CEP2 and 26% HCMD2/HPF2, or do the temp switch, then stock up a bit. With that setting the quad <bwdown> shows continuously above 500kBps... thank you Telecom/Alice.
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
The file to edit is client_state.xml and the value to change is bwdown in net_stats. Don't forget to shut down BOINC and make a backup of the file before you start. Thanks eviltoad! This did work for me and I was able to get the work to download.  It doesn't hold, though, as it slowly drops as WUs download. It's going to be a drag to have to do this for the entirety of the project run. I'm sure the admins have their reasons for the high bandwith limit, but I also wish it could be lowered. [Edit 1 times, last edit by Former Member at Jul 5, 2010 1:00:05 PM] |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Less than 2 days on, I'm getting:
06-Jul-2010 08:08:48 [World Community Grid] Message from server: The Clean Energy Project - Phase 2 requires 128.00 kbps download bandwidth. Your computer has been measured at 121.94 kbps That's with 1.5Mb/s ADSL and nothing else active on the network, so it looks like I'll have to edit the config file pretty much every second time I want to run this project.  |
||
|
|
Ingleside
Veteran Cruncher Norway Joined: Nov 19, 2005 Post Count: 974 Status: Offline Project Badges: 
|
Less than 2 days on, I'm getting: 06-Jul-2010 08:08:48 [World Community Grid] Message from server: The Clean Energy Project - Phase 2 requires 128.00 kbps download bandwidth. Your computer has been measured at 121.94 kbps That's with 1.5Mb/s ADSL and nothing else active on the network, so it looks like I'll have to edit the config file pretty much every second time I want to run this project. BOINC-client as default does 2 downloads per BOINC-project at once, and if you're getting multiple tasks at once, or each task includes multiple files needing to download, you'll download 2 files at once. Then client is doing two downloads at once, each download will max get roughly half of your total download-bandwidth, meaning for a 1.5 Mbit/s-connection each will max get 750 kbit/s instead. Since the limit for getting CEP2 is 1 Mbit/s, you've not got fast enough connection to handle two downloads at once. You can re-configure BOINC-client to only doing one download at once. Make, or edit if file already present, cc_config.xml (residing in your BOINC data-directory). The file should minimum include: <cc_config> To start using new option, in BOINC Manager's advanced view, select menu-option Advanced, Read config file.  "I make so many mistakes. But then just think of all the mistakes I don't make, although I might." |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Users should be aware as I commented a few days ago, that setting this value to 1 and having an issue with one download, will block all other downloading (and uploading separately) until the client gives up on that file DL/UL. Can work out as a rain into drip experience. Trial and error is advised.
----------------------------------------
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
Foundation
Cruncher Joined: Feb 22, 2009 Post Count: 4 Status: Offline Project Badges: 
|
I've got a real problem with this project, and have had to opt out for the time being. My network connection is ADSL - 3.5/1.5Mbps, but once one of these results files starts uploading, it becomes 3.5kbps/1.5Mbps. Turns out, if you saturate the upload, download speed on TCP/IP drops to the floor. The problem is that I have ~75 cores using this one connection, meaning that I need to upload a result every 7 minutes or so.
----------------------------------------I'd like to continue on this project, but need a way to throttle upload and only have this project running on a few of my machines. If anyone has any suggestions for how I can configure this, please. Also, I have some experience with ab initio QM calculations, and would ask the researchers if they have looked at all the ways possible to reduce the results files. I know we routinely generate 100MB+ output files, but only need to save a few hundred kB of that. Do you really need all 80MB of the result? Return the nuclear configuration and do a single point calculation to regenerate the rest only on the promising compounds! Thanks, Cliff <Edit> Searched around and found how to configure only one of the machines to this project. We'll see if the impact on the network is tolerable. That still leaves the questions above for the researchers. I'd like to do more! </Edit> [Edit 1 times, last edit by Foundation at Jul 15, 2010 6:10:39 AM] |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Hi,
----------------------------------------How many devices is that? Those that have a small pipe but multiple devices that could be up/downloading concurrently, may want to consider setting a connection schedule with a cache that is like 1.5-2 days and set the activity menu to Network based on preferences. Device 1 From 12:00-13:00 Device 2 From 13:00-14:00 Device 3 From 14:00-15:00 etc The time blocks could be alternated so for instance X devices get the even days of the week and the Y devices the uneven days. The duration to connect is of course dependent on how much time is needed to upload completed work and download new (the smallest part). On using alternate days method, caching probably needs to be increased to 3 days, so if there is a network issue for one/some/all devices, the CPUs don't fall idle. The Device network schedule blocks you could size to need. A single/dual core will need less time to clear daily production compared to a quad and bigger. Also the full/part-time crunching plays in as a device only contributing 12 hours a day wont be needing that much time as a 24/7 device. The cache still needs setting to X days to bridge the off-line periods, where BOINC will automatically adjust the requested amount of work based on the average hours per day that a device is running. You would have to work out the detail scheme of rotation but do ask if you're stuck figuring it out. A lead how to set scheduled networking with daily varying times is found in this FAQ: http://www.worldcommunitygrid.org/forums/wcg/viewthread?thread=20741 This optimizes utilization, improves your netspeed rating, but the price is delayed reporting. It's a wonder to be honest that any device by itself manages to get the 1Mbps measurement to qualify for CEP2 work if all are concurrently and randomly up/downloading in your present set up. Thanks for crunching at WCG. edit: The time blocks you might maybe slightly overlap in case there is the incidental extended upload to be done. So for instance 12:00-13:15, 13:00-14:15 etc.
WCG
----------------------------------------Please help to make the Forums an enjoyable experience for All! [Edit 2 times, last edit by Sekerob at Jul 15, 2010 12:34:29 PM] |
||
|
|
Ingleside
Veteran Cruncher Norway Joined: Nov 19, 2005 Post Count: 974 Status: Offline Project Badges:
|
I've got a real problem with this project, and have had to opt out for the time being. My network connection is ADSL - 3.5/1.5Mbps, but once one of these results files starts uploading, it becomes 3.5kbps/1.5Mbps. Turns out, if you saturate the upload, download speed on TCP/IP drops to the floor. The problem is that I have ~75 cores using this one connection, meaning that I need to upload a result every 7 minutes or so. I'd like to continue on this project, but need a way to throttle upload and only have this project running on a few of my machines. If anyone has any suggestions for how I can configure this, please. Using Sekerob's method will be a good solution, but you'll still have the problem that while upload is happening you'll kill download-speed. BOINC-client includes separate preferences for limiting upload- and download-speeds, so if you limits upload-speed to 80 - 90% or something of capasity, there should be enough "excess" bandwidth to also handle downloads. 1.5 Mbit/s is 187.5 kbyte/s, and 90% of this is 168.75 kbyte/s, while 80% is 150 kbyte/s. The actual bandwidth can be less, you can see http://www.worldcommunitygrid.org/forums/wcg/viewthread_thread,16067 for how to test the speed, and report your speed in that thread. BTW, if each result is 80 MB, and you get 150 kbyte/s upload-speed, you'll use 9.1 minutes per result. If all 75 cores runs CEP2, this will mean you'll use 11.4 hours total time to upload a result from each core... "I make so many mistakes. But then just think of all the mistakes I don't make, although I might." |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Was the bandwidth control part fixed as I seem to remember it was broken. If working again [let me test with 6.10.56 on Linux], an additional good way to play the upload restriction to get a better download rating... if both happen simultaneous, which on a 24/7 connection should be with few devices for most at default settings not the norm. From here it seems that after the US wakes up, the performance goes south i.e. time of day is a factor depending on where you are on the planet.
----------------------------------------
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
Foundation
Cruncher Joined: Feb 22, 2009 Post Count: 4 Status: Offline Project Badges:
|
If I understand Sekerob's proposal, it will fail to solve my problem. As touched on by Ingleside in his last comment, the output for this project from my computational plant exceeds my upload capability. To complete his calculation... My systems take between 5 and 10 hours for each CEPP2 WU, so I complete roughly 250 per day. Therefore, I would need >24hrs to transfer a day's results. Scheduling won't get around that!
Ingleside's upload throttling will solve the problem of crippling download while results are being uploaded. For the present, this is what I will do, with two or three of my faster machines (~16 cores) running CEPP2. But unless I restrict this project to this subset of my systems, I will have an ever increasing cache of results that are waiting to be transfered. I'd like to have someone from Harvard chime in and convince me they are being as efficient as possible with these transfers. How about tiering this another level, with bandwidth limited nodes only returning the nuclear configuration and those with big pipes returning the full calculation results and doing the single point calculations (if that is relevant to your ab initio QM problem). Regards, Cliff |
||
|
|
|