| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Active Research
Forum: Africa Rainfall Project
Thread: ARP downloads stalled...backing off...RESOLVED? |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 31
|
|
| Author |
|
|
bcavnaugh
Cruncher USA Joined: Nov 8, 2013 Post Count: 13 Status: Offline Project Badges: 
|
I Hope So,
I still have a lot of tasks waiting to download even after 9 hours The main website is also very slow and it takes 83 seconds to just download the image on the main page. |
||
|
|
Mike.Gibson
Ace Cruncher England Joined: Aug 23, 2007 Post Count: 12594 Status: Offline Project Badges: 
|
The situation with ARP downloads is ridiculous. ARP requires lots of files and some of the files are nearly 50 MB, so there is a huge amount to download before a unit can start.
If you get, say, a dozen units, a few files will download and then it stops. Some time later you get a few more files and so it goes on. The files downloaded are not from one unit at a time, so it can take hours before the first unit can start to be crunched. Meanwhile we are losing time from the deadlines. It is also not possible to get any more units because some are stalled. **********THIS IS A MAJOR PROBLEM!!!!!!********** Mike |
||
|
|
robertmiles
Senior Cruncher US Joined: Apr 16, 2008 Post Count: 445 Status: Offline Project Badges: 
|
It now appears the time limits are set when the list of files is downloaded, and not updated when the file downloads are slow. Another project has found that it is possible to change the time limit on the server but not on the clients. As a result, two of the last three ARP tasks assigned to my computer were finally completed by other computers before my computer could download all of the input files and those two tasks were cancelled by the server. My computer finally downloaded all of the input files for the third task and immediately started it.
|
||
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 2173 Status: Offline Project Badges: 
|
The main website is also very slow and it takes 83 seconds to just download the image on the main page. I definitely don't have and never had THAT problem...Ralf |
||
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 2173 Status: Offline Project Badges:
|
The situation with ARP downloads is ridiculous. ARP requires lots of files and some of the files are nearly 50 MB, so there is a huge amount to download before a unit can start My observation so far is not the actual transfer speed, but properly open up a connection. Those are two different issues.While download speeds can vary a lot due to a lot of different influences, with most of them outside of WCG, IF the BOINC client, I never had an issue to get that file download in one try. It always has been an issue of getting the connection successfully opened... Ralf |
||
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 2173 Status: Offline Project Badges:
|
It now appears the time limits are set when the list of files is downloaded, and not updated when the file downloads are slow. Another project has found that it is possible to change the time limit on the server but not on the clients. As a result, two of the last three ARP tasks assigned to my computer were finally completed by other computers before my computer could download all of the input files and those two tasks were cancelled by the server. My computer finally downloaded all of the input files for the third task and immediately started it. It always has been the case that the "clock starts" for a WU as soon as it is being downloaded. It's a matter of the BOINC client, not of WCG. In the past however, WCG techs were able to run manual scripts to give credit in cases where there was some kind of delay that would warrant this. In the current situation, we still have to wait to get normal operation established again.  Ralf |
||
|
|
Mike.Gibson
Ace Cruncher England Joined: Aug 23, 2007 Post Count: 12594 Status: Offline Project Badges:
|
Ralf
I agree that the download speeds are OK when it is actually downloading files. The issue is with the lack of overall ability to download all files one after the other (or rather 2 by 2). After downloading a couple of files it stops and it is ages before it attempts another download. I suspect that there is a very significant shortfall in bandwidth to cope with everyone's demand for work. Even pressing Retry takes for ever! The other problem is that the downloads do not occur on a work unit by work unit basis, so it can take hours before the first unit is available to be crunched. This was not a problem when downloading files was continuous. Then while these files are queuing, more work cannot be requested. Mike |
||
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 2173 Status: Offline Project Badges:
|
After downloading a couple of files it stops and it is ages before it attempts another download. I suspect that there is a very significant shortfall in bandwidth to cope with everyone's demand for work. Even pressing Retry takes for ever! As I mentioned before, it isn't really a "bandwidth" issue. It is an issue of getting an open connection, in order to be able to actually send anything. And in my experience, this is an issue on the sending server, which either can't handle enough concurrent connections or an issue with the underlying file system on that server/those servers...The other problem is that the downloads do not occur on a work unit by work unit basis, so it can take hours before the first unit is available to be crunched. This was not a problem when downloading files was continuous. There is no real issue here, as that is a symptom of the previous problem. The BOINC client, just with any BOINC project, just gets a list of files to download, it doesn't know (and doesn't care) what those files are and how they are connected. Some WCG projects are a bit extreme here, other projects just have one or two files per WU.And if problem No.1 is being finally resolved, this won't be an issue anymore. It just gets exaggerated right now because of the large number of really tiny files (in the larger picture of things), which under "normal circumstances" just download faster than you can follow the progress... Ralf |
||
|
|
Jim Slade
Veteran Cruncher Joined: Apr 27, 2007 Post Count: 669 Status: Offline Project Badges:
|
I cannot download data on my new computer. It says no work available to process
|
||
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 2173 Status: Offline Project Badges:
|
JSMH
|
||
|
|
|