| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Completed Research
Forum: The Clean Energy Project - Phase 2 Forum
Thread: CEP Uploads timing out (May 2) (resolved) |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 78
|
|
| Author |
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
snip quote:
(I do networking/firewalls/routers for a living, I know how to diagnose such an issue on my end) Excellent, Ralf... we just try to arrive as to where the bottleneck is. Got better ideas? Something getting saturated? It used to be 7 minutes from here for a _4 30Mb result file, with a rated 996KB upload speed ADSL2+. Looking back in my logs, it's currently about 25 minutes atm. From the message log entry where "Started upload..." is logged and where "Finished upload..." is logged (crunching 1 at the time). So what's different from Germany, versus from Italy... BND siphoning off borrowing a copy ;? BTW, Linux (11.10 and 12.04) and WLAN hate me doing this until I switch off the power management (trial and error). --//-- |
||
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 2173 Status: Offline Project Badges: 
|
snip quote: Well, yes, that the same problem that was mentioned before, an issue with the "round robin DNS" for the two upload servers (I think this was actually mentioned in another thread) is striking again...(I do networking/firewalls/routers for a living, I know how to diagnose such an issue on my end) Excellent, Ralf... we just try to arrive as to where the bottleneck is. Got better ideas? Something getting saturated? Looking back in my logs, it's currently about 25 minutes atm. From the message log entry where "Started upload..." is logged and where "Finished upload..." is logged (crunching 1 at the time). So what's different from Germany, versus from Italy... BND siphoning off borrowing a copy ;? Well, you seem to make a lot of strange assumption. Though I am German, I am living in the USA, urban Los Angeles area...And as I mentioned, it happens on two different setups, one at my place in the San Fernando valley, the other one in a Santa Monica business park.... BTW, Linux (11.10 and 12.04) and WLAN hate me doing this until I switch off the power management (trial and error). Neither WLAN nor Linux involved here...I run CEP2 on one Windows XP Pro SP3 machine and one 27" iMac at my home, and it is running on an underutilized, 3 month old quad Xeon server running Windows Server 2003, all hard-wired via Gigabit Ethernet LAN on broadband connections that are up 24/7/365 (unless SoCal Edison in SM is cutting some power lines or blows a transformer, again)... Ralf |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Holy Smack, Plowing through the the log archive, this was 2 days ago:
----------------------------------------22-May-2012 07:48:46 [World Community Grid] Started upload of E207775_028_C.28.C21H11N3S3Si.01246092.3.set1d06_0_4 22-May-2012 07:48:49 [World Community Grid] Finished upload of E207775_028_C.28.C21H11N3S3Si.01246092.3.set1d06_0_2 22-May-2012 07:48:51 [World Community Grid] Finished upload of E207775_028_C.28.C21H11N3S3Si.01246092.3.set1d06_0_1 22-May-2012 07:49:00 [World Community Grid] [checkpoint] result cfsw_1983_01983289_0 checkpointed 22-May-2012 07:50:28 [World Community Grid] [checkpoint] result cfsw_1987_01987912_0 checkpointed 22-May-2012 07:50:30 [World Community Grid] [checkpoint] result cfsw_1987_01987089_0 checkpointed 22-May-2012 07:50:55 [World Community Grid] [checkpoint] result cfsw_1983_01983029_0 checkpointed 22-May-2012 07:51:44 [World Community Grid] [checkpoint] result cfsw_1983_01983289_0 checkpointed 22-May-2012 07:52:50 [World Community Grid] [checkpoint] result cfsw_1987_01987912_0 checkpointed 22-May-2012 07:53:34 [World Community Grid] [checkpoint] result cfsw_1987_01987089_0 checkpointed 22-May-2012 07:53:39 [World Community Grid] Finished upload of E207775_028_C.28.C21H11N3S3Si.01246092.3.set1d06_0_4 Did not set flag to record the file size transfer, but remember it having taken 6 hours to crunch, when others do mostly 10-11 hours... The shorty will have been about 25Mb. Maybe it is the time of the day when it starts to have these X-Atlantic HD video streams, or just, BOINC disliking being watched ;-) --//-- edit: Of course, Harvard IT is the one who knows where it's coming from and how fast it ends on Jabba. [Edit 1 times, last edit by Former Member at May 24, 2012 5:11:15 PM] |
||
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 2173 Status: Offline Project Badges:
|
Here is the start of the logs regarding the upload of the WU, it just keeps repeating the last 6-7 lines in ever increasing intervals.
----------------------------------------5/24/2012 3:57:20 AM World Community Grid Computation for task E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0 finished 5/24/2012 3:57:20 AM World Community Grid Starting X0900063241410200602152328_1 5/24/2012 3:57:21 AM World Community Grid Starting task X0900063241410200602152328_1 using hcc1 version 642 5/24/2012 3:57:22 AM World Community Grid Started upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_0 5/24/2012 3:57:22 AM World Community Grid Started upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_1 5/24/2012 3:57:24 AM World Community Grid Finished upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_0 5/24/2012 3:57:24 AM World Community Grid Started upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_2 5/24/2012 3:57:27 AM World Community Grid Finished upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_2 5/24/2012 3:57:27 AM World Community Grid Started upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_3 5/24/2012 3:57:28 AM World Community Grid Finished upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_1 5/24/2012 3:57:28 AM World Community Grid Finished upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_3 5/24/2012 3:57:28 AM World Community Grid Started upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_4 5/24/2012 4:00:45 AM Project communication failed: attempting access to reference site 5/24/2012 4:00:45 AM World Community Grid Temporarily failed upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_4: HTTP error 5/24/2012 4:00:45 AM World Community Grid Backing off 1 min 0 sec on upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_4 5/24/2012 4:00:46 AM Internet access OK - project servers may be temporarily down. 5/24/2012 4:01:45 AM World Community Grid Started upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_4 5/24/2012 4:04:25 AM World Community Grid Temporarily failed upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_4: HTTP error 5/24/2012 4:04:25 AM World Community Grid Backing off 1 min 0 sec on upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_4 5/24/2012 4:05:25 AM World Community Grid Started upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_4 5/24/2012 4:06:41 AM World Community Grid [error] Error reported by file upload server: [E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_4] error locking file 5/24/2012 4:06:41 AM World Community Grid Temporarily failed upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_4: transient upload error 5/24/2012 4:06:41 AM World Community Grid Backing off 1 min 0 sec on upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_4 5/24/2012 4:07:42 AM World Community Grid Started upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_4 5/24/2012 4:12:55 AM Project communication failed: attempting access to reference site 5/24/2012 4:12:55 AM World Community Grid Temporarily failed upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_4: HTTP error 5/24/2012 4:12:55 AM World Community Grid Backing off 1 min 0 sec on upload of E207805_874_C.29.C21H10N6S2.00994348.2.set1d06_0_4 5/24/2012 4:12:57 AM Internet access OK - project servers may be temporarily down. [Edit 1 times, last edit by TPCBF at May 24, 2012 5:15:21 PM] |
||
|
|
Sgt.Joe
Ace Cruncher USA Joined: Jul 4, 2006 Post Count: 7848 Status: Recently Active Project Badges:
|
I also had trouble yesterday. Normally it takes about 25 minutes to upload a 25 mb file for me, but yesterday it was over 2 hours and stopped and restarted several times due to timeouts. But it did finally completely upload. Just speculating, but I think the problem is on the Harvard end.
----------------------------------------Cheers
Sgt. Joe
*Minnesota Crunchers* |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Never see those errors here [not the transient, not th locking, not the http error, except latter when the server was down last Sunday. Don't know the distance from SM to Toronto or Cambridge, but I'm pretty sure my connection is a whole lot longer, plus the trunk to pass from NL/UK to the AM side.
Maybe add the list of IP's to your hosts file.. that of WCG (5) & Harvard (2). It certainly improved connecting from here [speed & responsiveness]. Also the noted quick switching of DNS server has helped... alternating between Google and OpenDNS, completely ignoring my own ISP's auto assigned addresses [business.telecom]. --//-- |
||
|
|
kateiacy
Veteran Cruncher USA Joined: Jan 23, 2010 Post Count: 1027 Status: Offline Project Badges: 
|
Yesterday one CEP2 WU had an interrupted upload but finally completed. I just noticed it stuck in Transfers and didn't check the log for messages. Uploading from Iowa in US.
---------------------------------------- |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
I'm having lots of retries where the upload would reach 100% & then go to upload pending (retry xxx) stuck on 100%.
When I click on retry it would restart @ +/- 65%. Sometimes the upload only reaches only xx% then dies Sometimes the upload transfer will pause for a while & continue. This causes a repeat of data upload slowing down the overall upload. This behavior started a few day ago On an i7 with 7 (DistRTgen free a CPU core) WUs running the transfer box is always active with uploads In general the upload / web response time to WCG is very poor compared to other projects with big uploads like DistRen My ISP upload xfer limit is 55k that worked fine in the past Just posting this from a different pc took 3 attempts lol |
||
|
|
pcwr
Ace Cruncher England Joined: Sep 17, 2005 Post Count: 10903 Status: Offline Project Badges: 
|
My latest CEP2 wu wouldn't start uploading. After a reboot it uploaded.
----------------------------------------Win7 BOINC 6.10.58 Patrick  |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Hello cleanenergy,
We are getting multiple threads on this issue. See https://secure.worldcommunitygrid.org/forums/wcg/viewthread_thread,33205 Some sort of network congestion is causing timeouts on some CEP2 uploads to Harvard. People are noticing this. It eventually finishes, but the condition can last a good part of a day. You need to have your network people investigate this issue, so you can post that this problem is being investigated. People are patient about problems they know are being worked on. Lawrence |
||
|
|
|