| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Official Messages
Forum: News
Thread: 2022-10-27 Update (Workunits & storage update) |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 46
|
|
| Author |
|
|
Sgt.Joe
Ace Cruncher USA Joined: Jul 4, 2006 Post Count: 7846 Status: Offline Project Badges: 
|
Cyclops:
----------------------------------------Thanks for the update. Keep us informed. Cheers
Sgt. Joe
*Minnesota Crunchers* |
||
|
|
ThreadRipper
Veteran Cruncher Sweden Joined: Apr 26, 2007 Post Count: 1324 Status: Offline Project Badges: 
|
Thanks for the update!
----------------------------------------If the number of connections might be a problem, would it be possible to group several OPNG (and other projects' files) into one package (zip or something) that would then unzip upon download into several separate WUs. Then one could send 10 OPNG WUs at once in a single transfer instead of lots of small files separately that need to create a new HTTPS connection? Wishing best of luck with the issue. My machines are ready and running - waitng for stress-tests and breakthroughs :)  Join The International Team: https://www.worldcommunitygrid.org/team/viewTeamInfo.do?teamId=CK9RP1BKX1 AMD TR2990WX @ PBO, 64GB Quad 3200MHz 14-17-17-17-1T, RX6900XT @ Stock AMD 3800X @ PBO AMD 2700X @ 4GHz |
||
|
|
ADDIE2014
Cruncher Joined: Apr 13, 2019 Post Count: 33 Status: Offline Project Badges:
|
Thanks for the update cyclops, I have about 1/3 of my capacity (44 cores)running continuously now and will add in the rest as needed
|
||
|
|
Cyclops
Senior Cruncher Joined: Jun 13, 2022 Post Count: 295 Status: Offline |
Thanks for the update! If the number of connections might be a problem, would it be possible to group several OPNG (and other projects' files) into one package (zip or something) that would then unzip upon download into several separate WUs. Then one could send 10 OPNG WUs at once in a single transfer instead of lots of small files separately that need to create a new HTTPS connection? Wishing best of luck with the issue. My machines are ready and running - waitng for stress-tests and breakthroughs :) Hi ThreadRipper, thanks for your suggestion, I passed it on to the tech team. |
||
|
|
Grumpy Swede
Master Cruncher Svíþjóð Joined: Apr 10, 2020 Post Count: 2494 Status: Offline Project Badges: 
|
<Snip> @Cyclops: How can you possibly determine if the problem lies with OPNG, if you like now don't provide any OPNG's for downloading?Another (less optimistic) theory is that other tasks, specifically OPNG, were the cause of our recent storage issues and database-wide system errors. We have no solid evidence yet, only an observation that there is typically a decline in available OPNG work around the same time the download issues are less prevalent. A high load on the storage server and scheduler coincide with the database crashes and a phenomenon whereby the download/upload server groups intermittently register as down from the perspective of our load balancer. <Snip> By now you should have been able to notice, that without OPNG's, the website and the DB does not crash. So, start sending out OPNG's, and if the same problem comes back again, then you will know. [Edit 3 times, last edit by Grumpy Swede at Oct 28, 2022 6:14:59 PM] |
||
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 2173 Status: Offline Project Badges:
|
Thanks for the update! In a rare previous post from one of the techs it was kind of confirmed that the issue is NOT the number http(s) connections on the web server (then you wouldn't immediately get those 107 byte sized response back!), but the number of connections between web server and underlying database server...If the number of connections might be a problem, would it be possible to group several OPNG (and other projects' files) into one package (zip or something) that would then unzip upon download into several separate WUs. Then one could send 10 OPNG WUs at once in a single transfer instead of lots of small files separately that need to create a new HTTPS connection? Ralf |
||
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 2173 Status: Offline Project Badges:
|
well whaddya know?
Just right in time to ring in the weekend, download errors are back. (And no OPNG WUs to see anywhere, only OPN1 and a sprinkling of MCM1s) Ralf |
||
|
|
adriverhoef
Master Cruncher The Netherlands Joined: Apr 3, 2009 Post Count: 2346 Status: Offline Project Badges:
|
download errors are back. No sign of serious HTTP errors on my end. Had some at 01:31- 01:34 UTC and at 02:59 - 03:02 UTC, way too short to call it serious. |
||
|
|
Link64
Senior Cruncher Joined: Feb 19, 2021 Post Count: 206 Status: Offline Project Badges: 
|
Thanks for the update! If the number of connections might be a problem, would it be possible to group several OPNG (and other projects' files) into one package (zip or something) that would then unzip upon download into several separate WUs. Then one could send 10 OPNG WUs at once in a single transfer instead of lots of small files separately that need to create a new HTTPS connection? Wishing best of luck with the issue. My machines are ready and running - waitng for stress-tests and breakthroughs :) Hi ThreadRipper, thanks for your suggestion, I passed it on to the tech team. Another suggestion: make mcm1.dataset-sarc1.txt sticky, it's deleted every time when a computer runs out of MCM work and downloaded again when it gets next WUs. That''s over 100MB of unnecessarily transfered data every time. Milkyway and Einstein use that for reusable files, even a lot smaller ones, Milkyway uses that even for 4kB files.  [Edit 1 times, last edit by Link64 at Oct 29, 2022 11:01:34 AM] |
||
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 2173 Status: Offline Project Badges:
|
Well, for what it's worth (worse?), I just got a minute ago 35 OPNG WUs on my programming laptop. Nothing though in my two other GPU capable hosts.
Let's see if this indeed increases the likelyhood of download errors (the ones from last night seemed to be gone in the morning)... Ralf |
||
|
|
|