| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Support
Forum: Suggestions / Feedback
Thread: Bandwidth wasted by interrupted WUs downloads |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 18
|
|
| Author |
|
|
GolfSierra
Cruncher Joined: Jul 31, 2021 Post Count: 17 Status: Offline |
The bandwdith problem / overwhelmed download servers still remains an issue.
----------------------------------------I noticed that all DL from MCM and COVID19 are interrupted. Typically MCM (WU size 459 bytes) stops at 23,31 %, and COVID19 (WU size 1.02 KB) stops at 10.25 %. When trying to refresh the DL, the parts already sent are discarted and the DL starts again at 0%, is interruped again and so on. It would be required to look into the reasons, why the DL are interrupted instead of just postponed. There is a significant difference between a DL, that not starts and is postponed several times and a DL that is started, a small fraction of the WU is transmitted, and then discarted by the next DL refresh. Considering the amount of WU fragments that are sent out and then discarted, this is a enormous waste of bandwidth. It would be a wise move to just postpone DLs if the server ist too busy and allow a DL to be completed once it started. [Edit 1 times, last edit by GolfSierra at Oct 3, 2022 6:24:05 PM] |
||
|
|
rjs5
Cruncher Joined: Jan 22, 2011 Post Count: 6 Status: Offline Project Badges: 
|
I have noticed the problem with small files too.
I think the default network packet size is 4kb. It seems like it would be more efficient for a project to combine the small files before they are sent. That would save network connection attempts. |
||
|
|
davidak.de
Cruncher Joined: Feb 16, 2016 Post Count: 3 Status: Offline Project Badges: 
|
For me the download is interrupted after 0.1 KB.
I get these message in log: [World Community Grid] Started download of 73e84e94f6fc7fadccf24143b8926001.zip [World Community Grid] Temporarily failed download of 73e84e94f6fc7fadccf24143b8926001.zip: transient HTTP error [World Community Grid] Backing off 01:54:05 on download of 73e84e94f6fc7fadccf24143b8926001.zip This is very annoying. I hope someone is working on it. |
||
|
|
keputnam
Cruncher Joined: Aug 25, 2014 Post Count: 19 Status: Offline Project Badges: 
|
For the love of God
Will SOMEBODY configure the download server properly? I just had a batch take so long to download there was less than 30 hours to go deadline |
||
|
|
adriverhoef
Master Cruncher The Netherlands Joined: Apr 3, 2009 Post Count: 2173 Status: Offline Project Badges: 
|
GolfSierra:
----------------------------------------There is a significant difference between a DL, that not starts and is postponed several times and a DL that is started, a small fraction of the WU is transmitted, and then discarted by the next DL refresh. Actually, the small fraction that you say is downloaded is in fact not a small fraction of the file being downloaded. To repeat myself, it is not a small fraction, it is an error message. You can read about it in post 676279: "Yes, the response on a failed connection is a 503 error and that corresponds rather perfectly with those 107 bytes […] that you see as "transferred" when the connection actually doesn't get established." [Edit 1 times, last edit by adriverhoef at Oct 3, 2022 7:57:27 PM] |
||
|
|
GolfSierra
Cruncher Joined: Jul 31, 2021 Post Count: 17 Status: Offline |
Well, even if it is just that error message that is sent out by the DL servers, it still consumes some bandwidth which is lost for real WU transmissions.
----------------------------------------Just think of this happening some hundred times per minute, this adds up to a remarkable bandwidth consumption for nothing. [Edit 3 times, last edit by GolfSierra at Oct 3, 2022 8:14:50 PM] |
||
|
|
Eric Pohlke
Cruncher Canada Joined: Feb 4, 2006 Post Count: 15 Status: Offline Project Badges:
|
Starting on Oct 1, early afternoon, Completed Work Units stopped uploading and instead would be deferred. Then new Work Units stopped downloading right away and began to defer at 15-minute intervals. Today, the deferral intervals have moved up to 3 hours 15 minutes. My EPYC AMD 7700 now only gets work for a couple of hours per day. GPU Work Units are pretty much non-extant as the system only takes less than 10 seconds to do one. My connection to the World Community Grid website tends to crawl at times. Access anywhere else is not an issue on the 12-pipes x 1 Gbit connections it has to the internet.
Now you claim everything is complete and you’re back up and running. Yet, the connections and traffic from your server appears to be limited. |
||
|
|
KerSamson
Master Cruncher Switzerland Joined: Jan 29, 2007 Post Count: 1679 Status: Offline Project Badges: 
|
Considering the amount of WU fragments that are sent out and then discarded, this is a enormous waste of bandwidth. Absolutely !!! It is what frustrates me a lot. Not solving this particular problem kills any effort to participate. Ignoring this problem will simply cause WCG death because contributors will leave and never come back again. Yves |
||
|
|
astromatto
Cruncher Joined: May 26, 2007 Post Count: 4 Status: Offline Project Badges:
|
same here, interestingly enough it seems to match the same % I see locally. Downloading WUs is basically impossible, today, I just elaborated 5, I am trying to download new things since yesterday and 5 full tasks it's the only thing I can get.
Worst, whenever a download becomes, the max speed I get is 100 KB/S, but 99% of the WUs are downloaded at 5-7 KB/s Upload tho, seems pretty speedy, I see MB/s speeds |
||
|
|
|