Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 35
Posts: 35   Pages: 4   [ Previous Page | 1 2 3 4 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 6660 times and has 34 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: CEP2: Big size Work Units

Dear Saul Luizaga,
sorry to hear that the uploads get too big for you, but we understand that you prefer to pull out under these circumstances. Two tips which may make your life easier: 1) restrict your settings to only run 1-2 CEP2 wus at a time; 2) restrict your upload times to the middle of the night when you are not using your network bandwidth anyways.
Also: In a few weeks we'll get about 2 million smaller wus in again with significantly smaller uploads - maybe you can come back to CEP2 then.
Best wishes and happy holidays
Your Harvard CEP team
[Dec 30, 2011 8:48:07 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: CEP2: Big size Work Units

@Dark Angel: Thanks for the comfort message, yes in deed is demanding.

@cleanenergy: I have gave it another try recently and yes WU results are 20-21 MB, but I just saw a WU go from ~52% to 100% in an instant and started to upload. Why this premature ending is happening? isn't that incorrect?
[Feb 26, 2012 10:46:54 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: CEP2: Big size Work Units

Saul, the progres % is tuned to flat run 12 hours. No way [well, very difficult], to project completion on this science, lest they number of jobs done in the task were factored in]. Many of the jobs inside a task run shortish [5-10 minutes], but there are some that run multiple-to many hours. Sooo, yes, the jumping from 52% to 100% if the task does not need the full 12 hours is normal... bet you yours finished in just over 6 hours.

--//--

edit: Of course, not to forget those tasks that go out early with one of those RC= codes. No more mileage to achieve to continue [See result log link in the status column of respective WU].
----------------------------------------
[Edit 1 times, last edit by Former Member at Feb 27, 2012 1:57:38 PM]
[Feb 27, 2012 1:52:40 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: CEP2: Big size Work Units

Me (and I know the subject has been covered), I just wish BOINC would incorporate 7-zip on the uploads. I get radically unbalanced transfer usages because of one-way compression; e.g. last month I showed 21.2 GB download but 99.3 GB upload. And since my upload link is only 1/3rd the speed of my download link (which is rather common in the U.S. of A., at least)...

Again, I know the subject has been covered...but BOINC/WCG data isn't the only thing I upload hither and yon; be nice if it would be more...bandwidth-friendly?

Technically speaking, it would be easy enough to implement as an external queue to the individual projects running on its own thread that accepted destination URL and the data file path from any project and then calculated a SHA-256 (or something) checksum *hash for the before-compression data file and either packaged same into the compressed upload before doing the actual transfer or included the checksum in the transfer transaction.

Of course coordinating implementation across projects is a different story...which makes my wishes elusive fishes, indeed. crying

*Edit: Overwhelmed by urge for technical correctness.
----------------------------------------
[Edit 1 times, last edit by Former Member at Feb 27, 2012 6:19:37 PM]
[Feb 27, 2012 6:11:04 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: CEP2: Big size Work Units

Ok SekeRob, that's what they meant when they wrote that CEP2 WUs will be shorter on another thread. WUs took about 5 hours.

@ibsteve2u: your idea is good, I will suggest it to 7-Zip so they add it and unload that from every project.

BOINC should have a on the fly upgrade so if, like in this case, a better compression component will replace a old one it could be sent directly from BOINC developer center in Berkeley to the WCG & other project servers, that in turn they'd distribute them to the clients/servers, so the change would be completely transparent and only the projects would receive and e-mail announcing the change and they'd know how the filenames change from .gz to .7z for example and other details.

I see that this could be a problem with database management on the projects' servers where they have to match the BOINC filenames and/or compression method, so maybe this could be warned by BOINC a few weeks earlier and only be installed on projects that accept the change so if they don't they'll continue to using old components until they can catch up.

I will continue to participate since 20MB upload is not that big.
----------------------------------------
[Edit 1 times, last edit by Former Member at Mar 1, 2012 9:07:12 PM]
[Feb 28, 2012 2:07:33 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: CEP2: Big size Work Units

Hi ibsteve2u et al,
yes, we absolutely agree that 7z would make a lot of sense, but well - it is unfortunately not in our hands. Maybe the BOINC developers will make it available at some point.
Hi Saul Luizaga,
glad to hear that you'll stick around with CEP2 for a while biggrin !
Best wishes from
Your Harvard CEP team
[Mar 7, 2012 4:39:55 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Mumak
Senior Cruncher
Joined: Dec 7, 2012
Post Count: 477
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: CEP2: Big size Work Units

Still no hope for a 7zip compression? The upload data sizes bring my connection to knees... OK, I'm running 16 CEP2 WUs concurrently, but... smile

I made a comparison of result size:
Uncompressed: 61.2 MB
Compressed by BOINC (gzip?): 43.8 MB
Compressed using RAR (shareware): 32.5 MB
Compressed using 7zip/LZMA/Ultra (FREEWARE): 25.6 MB

Quite a difference...
----------------------------------------

----------------------------------------
[Edit 2 times, last edit by Mumak at Oct 14, 2013 1:32:42 PM]
[Oct 14, 2013 1:30:45 PM]   Link   Report threatening or abusive post: please login first  Go to top 
mali vuk
Advanced Cruncher
Slovenia
Joined: Apr 27, 2007
Post Count: 138
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: CEP2: Big size Work Units

There are no traffic problems in my country. My bill is also the same if my traffic is 0MB or 100GB. I learned that in other part in the world they are paying /transit. Special greeting to people who are paying more because of Boinc transit.

Please help them to decrease traffic. Also CO2 impact is also important.
----------------------------------------

[Oct 14, 2013 1:47:37 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: CEP2: Big size Work Units

Still no hope for a 7zip compression? The upload data sizes bring my connection to knees... ..

....Why ask here on WCG? You should be asking BOINC on their site. wink
[Oct 14, 2013 3:46:24 PM]   Link   Report threatening or abusive post: please login first  Go to top 
dango
Senior Cruncher
Joined: Jul 27, 2009
Post Count: 307
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: CEP2: Big size Work Units

Still no hope for a 7zip compression? The upload data sizes bring my connection to knees... OK, I'm running 16 CEP2 WUs concurrently, but... smile

I made a comparison of result size:
Uncompressed: 61.2 MB
Compressed by BOINC (gzip?): 43.8 MB
Compressed using RAR (shareware): 32.5 MB
Compressed using 7zip/LZMA/Ultra (FREEWARE): 25.6 MB

Quite a difference...


Mumak,

we in SVK have very good infrastructure of network; I don't know your worry. Based on your speed of computing there is ~1.5GB traffic/day of CEP2. It is not too much on our links. Other non-European countries can be jealous smile
----------------------------------------
[Edit 1 times, last edit by dango at Oct 14, 2013 5:21:09 PM]
[Oct 14, 2013 5:20:35 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 35   Pages: 4   [ Previous Page | 1 2 3 4 | Next Page ]
[ Jump to Last Post ]
Post new Thread