| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Completed Research
Forum: FightAIDS@Home Phase 2
Thread: Errors |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 37
|
|
| Author |
|
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
The download will have happened before you set the log flag and only when the task is started come the wrong bits to light (a guess). Strikes me as something needs fixing in the client, but a premature strike it is [the big 'Why' being on it only happening to FAH2]... we better get some backup before reporting over at alpha mail.
 |
||
|
|
Jim1348
Veteran Cruncher USA Joined: Jul 13, 2009 Post Count: 1066 Status: Offline Project Badges: 
|
OK, though I keep a short buffer (0.1 + 0.5 days), and that includes some ATLAS, but will try again.
----------------------------------------EDIT: Also the debug flag was set: 02-Jun-2016 10:12:35 [---] log flags: file_xfer, sched_ops, task, file_xfer_debug Which is well before the "sent time" listed as 6/3/16 03:17:33. Also, I see no record of it in BOINCTasks History. I wonder if I ever got it at all? [Edit 1 times, last edit by Jim1348 at Jun 3, 2016 2:56:34 PM] |
||
|
|
Dana Helgeson
Cruncher USA Joined: Dec 2, 2005 Post Count: 15 Status: Offline Project Badges: 
|
Here's a log entry I found in BOINC. Shows transient HTTP failure, says Internet access is OK, project may be down, etc. I'm sure no expert, but this looks like a problem on the WCG side of things.
5/31/2016 10:31:55 PM | World Community Grid | Sending scheduler request: To fetch work. 5/31/2016 10:31:55 PM | World Community Grid | Requesting new tasks for CPU 5/31/2016 10:32:01 PM | World Community Grid | Scheduler request completed: got 1 new tasks 5/31/2016 10:32:03 PM | World Community Grid | Started download of 72630fca694354d71ebe31473e27f433.param 5/31/2016 10:32:03 PM | World Community Grid | Started download of 791abddbe3abaac9236610ccb7248322.dat 5/31/2016 10:32:07 PM | World Community Grid | Finished download of 72630fca694354d71ebe31473e27f433.param 5/31/2016 10:32:07 PM | World Community Grid | Finished download of 791abddbe3abaac9236610ccb7248322.dat 5/31/2016 10:32:07 PM | World Community Grid | Started download of 0eee242d95def35610ffe205c48ff1a5.dat 5/31/2016 10:32:07 PM | World Community Grid | Started download of c0c8c4eb365f532dae6e7be71328b5e8.dat 5/31/2016 10:32:08 PM | World Community Grid | Finished download of 0eee242d95def35610ffe205c48ff1a5.dat 5/31/2016 10:32:08 PM | World Community Grid | Finished download of c0c8c4eb365f532dae6e7be71328b5e8.dat 5/31/2016 10:32:08 PM | World Community Grid | Started download of 0c663e860bfdf8007ea03e7ac1a388b2.dms 5/31/2016 10:32:08 PM | World Community Grid | Started download of c17c9055c4c18ef6471450f530575d50.dms 5/31/2016 10:32:13 PM | World Community Grid | Finished download of c17c9055c4c18ef6471450f530575d50.dms 5/31/2016 10:32:13 PM | World Community Grid | Started download of 50c94c01dfac96b158beec8cf70b4da9.rst 5/31/2016 10:32:18 PM | | Project communication failed: attempting access to reference site 5/31/2016 10:32:18 PM | World Community Grid | Temporarily failed download of 0c663e860bfdf8007ea03e7ac1a388b2.dms: transient HTTP error 5/31/2016 10:32:18 PM | World Community Grid | Finished download of 50c94c01dfac96b158beec8cf70b4da9.rst 5/31/2016 10:32:18 PM | World Community Grid | Started download of 483ef020cf8c6a03127862b4b14d5185.inp 5/31/2016 10:32:19 PM | World Community Grid | Started download of 0c663e860bfdf8007ea03e7ac1a388b2.dms 5/31/2016 10:32:19 PM | World Community Grid | Finished download of 483ef020cf8c6a03127862b4b14d5185.inp 5/31/2016 10:32:20 PM | | Internet access OK - project servers may be temporarily down. 5/31/2016 10:32:20 PM | World Community Grid | Finished download of 0c663e860bfdf8007ea03e7ac1a388b2.dms 5/31/2016 10:32:20 PM | World Community Grid | [error] MD5 check failed for 0c663e860bfdf8007ea03e7ac1a388b2.dms 5/31/2016 10:32:20 PM | World Community Grid | [error] expected 046ba4471aa324d0f29fac3c2faaa167, got 10222d8ae331923eb7598171cddd9169 5/31/2016 10:32:20 PM | World Community Grid | [error] Checksum or signature error for 0c663e860bfdf8007ea03e7ac1a388b2.dms |
||
|
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
Perfect example of what others had described... the download gets interrupted, resumes and [as one would expect], MD5 is immediately alerted on if wrong bits snug in during transition (The off-bits will be where the pause/resume point is.)
Why the client not refetching? Is the server not aware for the need to push a new copy? This is a client 'notice' worth message as bad job(s) sit on the client. |
||
|
|
Jim1348
Veteran Cruncher USA Joined: Jul 13, 2009 Post Count: 1066 Status: Offline Project Badges:
|
It looks like I am seeing the same thing, though it is intermingled with an MCM1 task.
06-Jun-2016 22:35:54 [World Community Grid] Sending scheduler request: To fetch work. 06-Jun-2016 22:36:05 [World Community Grid] Started download of fahb.FAH2_000084_avx38747_000090-in1.dms 06-Jun-2016 22:36:05 [World Community Grid] [file_xfer] URL: http://bdd7.http.cdn.softlayer.net/80BDD7/gri...4_avx38747_000090-in1.dms 06-Jun-2016 22:36:05 [World Community Grid] Started download of fahb.FAH2_000084_avx38747_000090-in2.dms 06-Jun-2016 22:36:05 [World Community Grid] [file_xfer] URL: http://bdd7.http.cdn.softlayer.net/80BDD7/gri...4_avx38747_000090-in2.dms 06-Jun-2016 22:36:06 [World Community Grid] [file_xfer] http op done; retval 0 (Success) 06-Jun-2016 22:36:06 [World Community Grid] [file_xfer] file transfer status 0 (Success) 06-Jun-2016 22:36:06 [World Community Grid] Finished download of fahb.FAH2_000084_avx38747_000090-in2.dms 06-Jun-2016 22:36:06 [World Community Grid] [file_xfer] Throughput 907146 bytes/sec 06-Jun-2016 22:36:06 [World Community Grid] Started download of 2959f2ca72ed351708e2e3aa1e47e026.rst 06-Jun-2016 22:36:06 [World Community Grid] [file_xfer] URL: https://grid.worldcommunitygrid.org/boinc/dow...d351708e2e3aa1e47e026.rst 06-Jun-2016 22:36:07 [World Community Grid] [file_xfer] http op done; retval -184 (transient HTTP error) 06-Jun-2016 22:36:07 [World Community Grid] [file_xfer] http op done; retval 0 (Success) 06-Jun-2016 22:36:07 [World Community Grid] [file_xfer] file transfer status -184 (transient HTTP error) 06-Jun-2016 22:36:07 [World Community Grid] Temporarily failed download of fahb.FAH2_000084_avx38747_000090-in1.dms: transient HTTP error 06-Jun-2016 22:36:07 [World Community Grid] [file_xfer] file transfer status 0 (Success) 06-Jun-2016 22:36:07 [World Community Grid] Finished download of 2959f2ca72ed351708e2e3aa1e47e026.rst 06-Jun-2016 22:36:07 [World Community Grid] [file_xfer] Throughput 246966 bytes/sec 06-Jun-2016 22:36:07 [World Community Grid] Started download of 6900f958cf92f47deb580b115ffc3094.inp 06-Jun-2016 22:36:07 [World Community Grid] [file_xfer] URL: https://grid.worldcommunitygrid.org/boinc/dow...2f47deb580b115ffc3094.inp 06-Jun-2016 22:36:07 [World Community Grid] Started download of MCM1_0123605_0070_MCM1_0123605_0070.txt 06-Jun-2016 22:36:07 [World Community Grid] [file_xfer] URL: https://grid.worldcommunitygrid.org/boinc/dow...070_MCM1_0123605_0070.txt 06-Jun-2016 22:36:08 [World Community Grid] [file_xfer] http op done; retval 0 (Success) 06-Jun-2016 22:36:08 [World Community Grid] [file_xfer] http op done; retval 0 (Success) 06-Jun-2016 22:36:08 [World Community Grid] [file_xfer] file transfer status 0 (Success) 06-Jun-2016 22:36:08 [World Community Grid] Finished download of 6900f958cf92f47deb580b115ffc3094.inp 06-Jun-2016 22:36:08 [World Community Grid] [file_xfer] Throughput 11115 bytes/sec 06-Jun-2016 22:36:08 [World Community Grid] [file_xfer] file transfer status 0 (Success) 06-Jun-2016 22:36:08 [World Community Grid] Finished download of MCM1_0123605_0070_MCM1_0123605_0070.txt 06-Jun-2016 22:36:08 [World Community Grid] [file_xfer] Throughput 6756 bytes/sec 06-Jun-2016 22:36:08 [World Community Grid] Started download of MCM1_0123605_7961_MCM1_0123605_7961.txt 06-Jun-2016 22:36:08 [World Community Grid] [file_xfer] URL: https://grid.worldcommunitygrid.org/boinc/dow...961_MCM1_0123605_7961.txt 06-Jun-2016 22:36:08 [World Community Grid] Started download of MCM1_0123605_8694_MCM1_0123605_8694.txt 06-Jun-2016 22:36:08 [World Community Grid] [file_xfer] URL: https://grid.worldcommunitygrid.org/boinc/dow...694_MCM1_0123605_8694.txt 06-Jun-2016 22:36:09 [World Community Grid] [file_xfer] http op done; retval 0 (Success) 06-Jun-2016 22:36:09 [World Community Grid] [file_xfer] http op done; retval 0 (Success) 06-Jun-2016 22:36:09 [World Community Grid] [file_xfer] file transfer status 0 (Success) 06-Jun-2016 22:36:09 [World Community Grid] Finished download of MCM1_0123605_7961_MCM1_0123605_7961.txt 06-Jun-2016 22:36:09 [World Community Grid] [file_xfer] Throughput 11813 bytes/sec 06-Jun-2016 22:36:09 [World Community Grid] [file_xfer] file transfer status 0 (Success) 06-Jun-2016 22:36:09 [World Community Grid] Finished download of MCM1_0123605_8694_MCM1_0123605_8694.txt 06-Jun-2016 22:36:09 [World Community Grid] [file_xfer] Throughput 12217 bytes/sec 06-Jun-2016 22:36:09 [World Community Grid] Started download of MCM1_0123605_0709_MCM1_0123605_0709.txt 06-Jun-2016 22:36:09 [World Community Grid] [file_xfer] URL: https://grid.worldcommunitygrid.org/boinc/dow...709_MCM1_0123605_0709.txt 06-Jun-2016 22:36:09 [World Community Grid] Started download of MCM1_0123605_9601_MCM1_0123605_9601.txt 06-Jun-2016 22:36:09 [World Community Grid] [file_xfer] URL: https://grid.worldcommunitygrid.org/boinc/dow...601_MCM1_0123605_9601.txt 06-Jun-2016 22:36:10 [---] Project communication failed: attempting access to reference site 06-Jun-2016 22:36:10 [World Community Grid] [file_xfer] http op done; retval 0 (Success) 06-Jun-2016 22:36:10 [World Community Grid] [file_xfer] http op done; retval 0 (Success) 06-Jun-2016 22:36:10 [World Community Grid] [file_xfer] file transfer status 0 (Success) 06-Jun-2016 22:36:10 [World Community Grid] Finished download of MCM1_0123605_0709_MCM1_0123605_0709.txt 06-Jun-2016 22:36:10 [World Community Grid] [file_xfer] Throughput 10810 bytes/sec 06-Jun-2016 22:36:10 [World Community Grid] [file_xfer] file transfer status 0 (Success) 06-Jun-2016 22:36:10 [World Community Grid] Finished download of MCM1_0123605_9601_MCM1_0123605_9601.txt 06-Jun-2016 22:36:10 [World Community Grid] [file_xfer] Throughput 11508 bytes/sec 06-Jun-2016 22:36:10 [World Community Grid] Started download of MCM1_0123605_7320_MCM1_0123605_7320.txt 06-Jun-2016 22:36:10 [World Community Grid] [file_xfer] URL: https://grid.worldcommunitygrid.org/boinc/dow...320_MCM1_0123605_7320.txt 06-Jun-2016 22:36:11 [---] Internet access OK - project servers may be temporarily down. 06-Jun-2016 22:36:11 [World Community Grid] [file_xfer] http op done; retval 0 (Success) 06-Jun-2016 22:36:11 [World Community Grid] Started download of fahb.FAH2_000084_avx38747_000090-in1.dms 06-Jun-2016 22:36:11 [World Community Grid] [file_xfer] URL: https://grid.worldcommunitygrid.org/boinc/dow...4_avx38747_000090-in1.dms 06-Jun-2016 22:36:11 [World Community Grid] [file_xfer] file transfer status 0 (Success) 06-Jun-2016 22:36:11 [World Community Grid] Finished download of MCM1_0123605_7320_MCM1_0123605_7320.txt 06-Jun-2016 22:36:11 [World Community Grid] [file_xfer] Throughput 6756 bytes/sec 06-Jun-2016 22:36:12 [World Community Grid] [file_xfer] http op done; retval 0 (Success) 06-Jun-2016 22:36:12 [World Community Grid] [file_xfer] file transfer status 0 (Success) 06-Jun-2016 22:36:12 [World Community Grid] Finished download of fahb.FAH2_000084_avx38747_000090-in1.dms 06-Jun-2016 22:36:12 [World Community Grid] [file_xfer] Throughput 32400 bytes/sec 06-Jun-2016 22:36:12 [World Community Grid] MD5 check failed for fahb.FAH2_000084_avx38747_000090-in1.dms 06-Jun-2016 22:36:12 [World Community Grid] expected 2e5543db57114d74a926946bb72ce20d, got 487bf772bb3b0babafd934ac1efcf84a 06-Jun-2016 22:36:12 [World Community Grid] Checksum or signature error for fahb.FAH2_000084_avx38747_000090-in1.dms |
||
|
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
I've raised a ticket on the Alpha mail list for council.
If this is a bug on the client side, it would be nice to learn what the client versions are and on what OSses they are running, just in case. |
||
|
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
A reply from a very experienced BOINCer:
Or, third option, a server deployment error. I've come across it before, when a project admin has changed the size/content of a file after deployment, but not changed the name or app_version record. That's documented at http://boinc.berkeley.edu/trac/wiki/BoincFiles |
||
|
|
Jim1348
Veteran Cruncher USA Joined: Jul 13, 2009 Post Count: 1066 Status: Offline Project Badges:
|
If this is a bug on the client side, it would be nice to learn what the client versions are and on what OSses they are running, just in case. BOINC 7.6.33 on Win7 64-bit for me, above. |
||
|
|
SekeRob
Master Cruncher Joined: Jan 7, 2013 Post Count: 2741 Status: Offline |
Here a further expert analysis, and some mechanics, the suggestion to reset the project, WCG, if the bad MD5 is reproducable, but I cant escape the impression that the root cause solution is once more in WCGs distribution system.
Dana Helgeson in your linked thread has the clue. 5/31/2016 10:32:20 PM | World Community Grid | Finished download of 0c663e860bfdf8007ea03e7ac1a388b2.dms 5/31/2016 10:32:20 PM | World Community Grid | [error] MD5 check failed for 0c663e860bfdf8007ea03e7ac1a388b2.dms 5/31/2016 10:32:20 PM | World Community Grid | [error] expected 046ba4471aa324d0f29fac3c2faaa167, got 10222d8ae331923eb7598171cddd9169 5/31/2016 10:32:20 PM | World Community Grid | [error] Checksum or signature error for 0c663e860bfdf8007ea03e7ac1a388b2.dms The 'expected' checksum is the MD5 for the file, calculated and stored on the server, transmitted in the <file> block in the sched_reply, and can be found in the client_state.xml file on the user's computer. The 'got' checksum is the MD5 calculated by the BOINC client from the received data, at the end of the download. It can be re-checked with any standard MD5 tool. They need to match. If they don't, I would troubleshoot in the following order. 1) Find the downloaded file on the user's disk, and re-generate MD5 manually. 99.999% certain it will match 'got', but let's be sure. 2) Download a fresh copy from the url in client_state, and repeat (1). 99.999% certain the outcome will be the same (the internet is pretty reliable these days). 3) Focus now on the 'expected' value in client_state. Get a new app_version record, perhaps by resetting the project in the client, or attaching a fresh host. If a fresh allocation for the application (not just a new task) gets the 'bad' MD5 shown by Dana, ask the project admins to check the value in their database and correct if needed. If a fresh allocation of the application gets a good MD5, advise affected users to reset the project asap. Slightly different versions of that procedure apply, depending on whether this is an application file or a task file, but I hope the general principle is clear |
||
|
|
Jim1348
Veteran Cruncher USA Joined: Jul 13, 2009 Post Count: 1066 Status: Offline Project Badges:
|
By the time we see the errors in the WCG Results Status, aren't those files long gone on our PCs? Maybe I am missing something.
|
||
|
|
|