| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
| World Community Grid Forums
|
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 98
|
|
| Author |
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 2173 Status: Offline Project Badges:
|
I have no confirmation on the subject but the 107 bytes downloaded before a download failed could only be the size of the error message corresponding to "connection unavailable" or "page not found" You don't really read and try to understand most of the forum posts, don't you?It is the complete text of the error message "503 Service Unavailable....." And again, a 5xx http(s) error is a "server side" error message, send back from the server, in contrast to a 4xx error code, which is a "client side" error code and generated by the http(s) client itself. This is well define http(s)/Internet traffic behavior... https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html https://httpwg.org/specs/rfc9110.html#status.503 Ralf |
||
|
|
HyperComputing
Advanced Cruncher Joined: Aug 10, 2019 Post Count: 74 Status: Offline Project Badges:
|
So in my apache config file all lines referring to error 4xx are useless ...
----------------------------------------
My GPU compute WUs as fast as I'm crushing bubble wrap.
|
||
|
|
mxplm
Cruncher Joined: Sep 12, 2009 Post Count: 13 Status: Offline Project Badges:
|
Yes TPCBF, HyperComputing is right, the HTTP 4xx status codes are also sent by the server. E.g. the server decides that the client sent a bad request and returns that as an answer to the client.
|
||
|
|
adriverhoef
Master Cruncher The Netherlands Joined: Apr 3, 2009 Post Count: 2346 Status: Offline Project Badges:
|
and no idea where that wcg "command" would be if it even exists, maybe I missed where it's referenced originally. Well, PMH_UK literally said: "wcgresults -x". Have you ever tried using the Search function on the forum, AlmightyMoe? Then try looking for wcgresults. It may work wonders realizing that you don't need the Retry Now button anymore, even when you are sleeping (using crontab)! I read that you use Linux, AlmightyMoe: Using 2 linux installs ... so you might wanna give it a try. |
||
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 2173 Status: Offline Project Badges:
|
Yes TPCBF, HyperComputing is right, the HTTP 4xx status codes are also sent by the server. E.g. the server decides that the client sent a bad request and returns that as an answer to the client. Did you ever bother to read the links? RFC9110 (which replaced RFC2616) is a pretty good start...Ralf |
||
|
|
Greg_BE
Advanced Cruncher Joined: May 9, 2016 Post Count: 124 Status: Offline Project Badges:
|
But shouldn't this issue be solved by the project?
I mean they keep bragging about how the science did this and that. Then comes the website being down, 30 minutes, oh no wait, longer and longer. Then we keep fighting these transient errors. This is getting really really old. To get something like 5 tasks to load all their parts, I have to slam the server manually over and over until all the files get downloaded. I don't sit here waiting to see, I just check randomly if anything is downloading and mostly before I shut down for the night. It took me 15 tries to get all the parts downloaded. |
||
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 2173 Status: Offline Project Badges:
|
But shouldn't this issue be solved by the project? Web site&forum on one side and the BOINC facing servers should be (and in the past at IBM, were) completely different setups. One going down (be it underlying database issues, upgrade, maintenance) should not take down the other part...I mean they keep bragging about how the science did this and that. Then comes the website being down, 30 minutes, oh no wait, longer and longer. Then we keep fighting these transient errors. And yes, now in the 6th month since resuming uploading WUs, this should be fixed by now. Unfortunately, due to the lack of communication, we don't know what is causing the delay in fixing this for good. It seems that at least a couple of times (just check Cyclops' overly optimistic announcement posts), they thought that this was done. But then it is impossible to tell if and how "the tech team" is getting its feedback. It certainly doesn't provide any feedback about what the actual issues are and what they a trying to do to fix it. So that they can get qualified feedback. Instead, over and over again, people keep riding on non-issues and/or come up with yet new ways to work around those issues FOR THEM, AND THEM ALONE, with some of their remedies making things more likely even worse... This is getting really really old. Yes, it does.Ralf |
||
|
|
HyperComputing
Advanced Cruncher Joined: Aug 10, 2019 Post Count: 74 Status: Offline Project Badges:
|
To get something like 5 tasks to load all their parts, I have to slam the server manually over and over until all the files get downloaded. I don't sit here waiting to see, I just check randomly if anything is downloading and mostly before I shut down for the night. It took me 15 tries to get all the parts downloaded. you can create a crontask with "/usr/bin/boinccmd --network_available" to do it for you. no need to be root for this command with a delay of 10 seconds all my download list is done in 15 minutes unfortunately all my OPNG list is also done in 15 minutes and my boinc client has to wait for the next list of WUs to download and I don't do this to have nice statistics but to return the maximum of results
My GPU compute WUs as fast as I'm crushing bubble wrap.
|
||
|
|
mxplm
Cruncher Joined: Sep 12, 2009 Post Count: 13 Status: Offline Project Badges:
|
Did you ever bother to read the links? No, I didn't. This is basic stuff for my daily work, I don't need to look it up anymore. RFC9110 (which replaced RFC2616) is a pretty good start... Thanks for the reference, it says that status codes are sent by the server, and that hasn't changed since HTTP 1.0 in 1996. First some basics: An HTTP "client" is a program that establishes a connection to a server for the purpose of sending one or more HTTP requests. An HTTP "server" is a program that accepts connections in order to service HTTP requests by sending HTTP responses. Regarding status codes in general: The status code of a response is a three-digit integer code [...] Note: There are no "request status codes" or "client status codes" in HTTP. And for the client-side error: The 4xx (Client Error) class of status code indicates that the client seems to have erred. Except when responding to a HEAD request, the server SHOULD send [...] an explanation of the error situation, and whether it is a temporary or permanent condition. Note also that man HTTP 4xx status codes cannot logically be given by the client: - HTTP 401 Unauthorized: The server decides who is allowed in. - HTTP 404 Not Found: How would a client know if a server has a certain site/resource without asking it? If you still don't believe me, google it. You will find the same info everywhere. Back to topic: On my machine, a script checks if there are unfinished transfers every few seconds, no matter if they are stalled. If so, the script calls boinccmd --file_transfer http://www.worldcommunitygrid.org/ <random_transfer_file> retry. If the maximum amount of transfers (default=2) are already in progress, this does nothing except clearing any project back-off timer, and putting a random file next in line to be tried. If there are less than 2 downloads running and there are stalled ones, the retry reactivates one or two of them. Without the script, I often don't get my WUs in time before the ones I have are done and my CPU is idle. This is mostly because the back-off timers quickly jump to waiting hours after a few failed download attempts. Also, while downloads are stalled the client won't ask for new work. With my script, the downloads are processed slowly but steadily and I never run out of WUs. Being aware that flooding the WCG servers with too many requests is not nice, I analyze the BOINC logs (file_xfer_debug option active) and the client does an average 2.5 requests per minute while downloading WUs, and none otherwise. I think for 32 threads running WUs that's a tolerable number. Around 50% of the requests are successful, meaning that on average each download fails only once before finishing successfully. |
||
|
|
HyperComputing
Advanced Cruncher Joined: Aug 10, 2019 Post Count: 74 Status: Offline Project Badges:
|
Downloads are 2 files by 2 files
----------------------------------------under normal conditions a download is done in 1 second under current conditions it takes 3 seconds for a download to be in error after 4 downloads in error all the others files go on pending time the "boinccmd --network_available" command allows the boinc client to resume where it left off without waiting for the pending time to expire it takes 6 seconds for the whole list to be pending, so a 10 second delay for the boinccmd command isn't too short 4 requests every 10 seconds vs. 1 every second, I don't think the server will crash for that. The goal is to return the most results even with a buggy server There are already very competent teams who are dealing with the various problems of downloading and slowness of the site.
My GPU compute WUs as fast as I'm crushing bubble wrap.
|
||
|
|
|