| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Active Research
Forum: Africa Rainfall Project
Thread: BIG file size! Hope you have a lot of bandwidth available |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 49
|
|
| Author |
|
|
bicotz
Advanced Cruncher Canada Joined: Apr 25, 2010 Post Count: 67 Status: Offline Project Badges: 
|
Had a look a the system requirements:
----------------------------------------https://www.worldcommunitygrid.org/help/viewTopic.do?shortName=minimumreq Per Workunit Download: 100MB Per Workunit Upload: 60MB Are these numbers right?   [Edit 1 times, last edit by bicotz at Oct 30, 2019 5:36:29 PM] |
|||
|
||||
|
mdxi
Advanced Cruncher Joined: Dec 6, 2017 Post Count: 109 Status: Offline Project Badges: 
|
Yes. All the numbers the team has published are in line with what I saw during the beta.
----------------------------------------The file sizes make sense, as each WU represents 48h of a climate model run over a specific geographic region, and the output of each WU is the input for the next run for that same region.  |
||
|
|
Dataman
Ace Cruncher Joined: Nov 16, 2004 Post Count: 4865 Status: Offline Project Badges: 
|
Yes, big bandwidth requirement. With as little as 3 wu's uploading at the same time they completely choked my network and take as long as 20 min. to upload. I had to limit my farm to only two machines and only 2 wu's/machine in order to do anything else. Reminds me of the clean water project which also had big output files.
----------------------------------------Otherwise they run quite well and can even be run concurrently with VBox projects provided one has lots of RAM.   |
||
|
|
dcs1955
Veteran Cruncher USA Joined: May 24, 2016 Post Count: 668 Status: Offline Project Badges:
|
Thanks to all the insight on this new project.. I am new at decripting the processing req for projects. Besides the large file sizes, does anyone know if continious interconnection is req for handshakes ala FAHB? That restriction has reduced my ability to crunch that project.
---------------------------------------- |
||
|
|
Sgt.Joe
Ace Cruncher USA Joined: Jul 4, 2006 Post Count: 7662 Status: Offline Project Badges: 
|
I see the first 16 units have been reported. They have averaged a bit over 14.5 hours in run time. I speculate that it is the more powerful machines which have accomplished this quick turnaround. Therefore I am thinking this average may increase a bit over time as some of the less powerful machines begin reporting their results. Let's watch and see.
----------------------------------------Cheers Edit: Spelling
Sgt. Joe
----------------------------------------*Minnesota Crunchers* [Edit 1 times, last edit by Sgt.Joe at Oct 31, 2019 1:14:38 AM] |
||
|
|
Speedy51
Veteran Cruncher New Zealand Joined: Nov 4, 2005 Post Count: 1290 Status: Offline Project Badges: 
|
Reminds me of the clean water project which also had big output files. If I remember correctly the "Clean Water Project" had reasonably small output files. I have a feeling you are talking about "Clean Energy Project Phase 2"  [Edit 1 times, last edit by Speedy51 at Oct 31, 2019 2:28:14 AM] |
||
|
|
Dataman
Ace Cruncher Joined: Nov 16, 2004 Post Count: 4865 Status: Offline Project Badges:
|
Reminds me of the clean water project which also had big output files. If I remember correctly the "Clean Water Project" had reasonably small output files. I have a feeling you are talking about "Clean Energy Project Phase 2" You are correct. Knew it had "Clean" in it.  |
||
|
|
JcJet
Cruncher Joined: Oct 5, 2010 Post Count: 3 Status: Offline Project Badges: 
|
No big deal.
It's notmal that data size rise with time as new projects apearing. And it's acutually means GOOD things. For science, not for someone's cellular internet. Because in science, the method or algorithm itself it's only half of the possible solution, the other half is dataset you use to 'mine' for new data. And if datasets are too small or have flaws like biases, it will affect result a lot. There are fields which stronlgy limited by the lack of bigger, broader datasets (even though the current ones are several TBs in size). So bigger datasets = good. |
||
|
|
Greg L
Advanced Cruncher Joined: Mar 2, 2007 Post Count: 94 Status: Offline Project Badges:
|
Not overly computer savvy so bear with me...
----------------------------------------Ok, they're big files. Other than the 1-2 times per day that they spend a few minutes down/uploading the files, is the project any different crunching wise than any other project? I'm not really seeing the issue other than the fact that you're dealing with big files (ok, if I was crunching on a smart phone & had a limited data plan...yeah, that could be an issue). On the computer I'm on now there are six cores and plenty of ram available & I usually have the Grid grinding away in the background with no issues. Would making say three of those cores available for the Africa project impact the computer's performance? Other than a lower 'Results Returned' score, 24 hours of crunching is 24 hours of crunching... or am I missing something here?  |
||
|
|
Jack007
Master Cruncher CANADA Joined: Feb 25, 2005 Post Count: 1604 Status: Offline Project Badges:
|
Finally got my first 2 work units.
----------------------------------------Took about a min apiece, but i'm sure that the hold up is at WCGs end not mine. I have 320 mbps down, and 16 mbps up. (just ran speed test) Wish I could get the 5 units, my machine is going to eat these up.  |
||
|
|
|