Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 35
Posts: 35   Pages: 4   [ Previous Page | 1 2 3 4 ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 6653 times and has 34 replies Next Thread
mmstick
Senior Cruncher
Joined: Aug 19, 2010
Post Count: 151
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: CEP2: Big size Work Units

You can always just put the 7zip archive inside of the gzip BOINC archive. It doesn't really matter what BOINC uses so long as your app just adds an extra stage to decompress the 7zip archive and compress the results before it gets packaged and sent back to BOINC. I would bet that a lot of people refuse to get on this project because of the large bandwidth requirement. There are quite a lot of people with nice computers but live in an area with bandwidth caps.
[Oct 29, 2013 9:16:17 AM]   Link   Report threatening or abusive post: please login first  Go to top 
mmstick
Senior Cruncher
Joined: Aug 19, 2010
Post Count: 151
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: CEP2: Big size Work Units

Actually, how could we get BOINC developers to switch to using xz instead of gz? It uses 7zip's lzma and provides it through the liblzma library. I know some Linux distros package their packages and Linux kernel with tar.xz instead of tar.gz these days since it produces significantly smaller file sizes, thus cutting down on bandwidth and disk usage costs.
[Nov 18, 2013 1:34:49 PM]   Link   Report threatening or abusive post: please login first  Go to top 
armstrdj
Former World Community Grid Tech
Joined: Oct 21, 2004
Post Count: 695
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: CEP2: Big size Work Units

It could be a licensing issue not sure of the license it uses. You might try discussing on the BOINC message boards.

Thanks,
armstrdj
[Nov 21, 2013 3:28:02 PM]   Link   Report threatening or abusive post: please login first  Go to top 
mmstick
Senior Cruncher
Joined: Aug 19, 2010
Post Count: 151
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: CEP2: Big size Work Units

It can't be a licensing issue since on their website it states:

Licensing

The most interesting parts of XZ Utils (e.g. liblzma) are in the public domain. You can do whatever you want with the public domain parts.

Some parts of XZ Utils (e.g. build system and some utilities) are under different free software licenses such as GNU LGPLv2.1, GNU GPLv2, or GNU GPLv3.


Which makes it no different than gz.

Some nice charts on compression statistics here: http://pokecraft.first-world.info/wiki/Quick_..._LZMA_vs_XZ_vs_LZ4_vs_LZO
----------------------------------------
[Edit 2 times, last edit by mmstick at Nov 22, 2013 8:34:32 AM]
[Nov 22, 2013 8:27:56 AM]   Link   Report threatening or abusive post: please login first  Go to top 
widdershins
Veteran Cruncher
Scotland
Joined: Apr 30, 2007
Post Count: 677
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: CEP2: Big size Work Units

Those charts are interesting reading. From looking at them it's not difficult to see why gz is probably preferred by those running boinc projects. Low memory use and fast compress/decompress times.

Whilst from our POV .gz may be less desirable due to the larger file sizes, at the far end where the servers are the additional memory and processor load on them from compressing and decompressing millions of WU's would probably rule out most other compression methods.

I have no doubt IBM could scale up its systems to cope, but unfortunately there are many other projects running on BOINC and I suspect many of those are run on a shoestring budget and would not have the resources to beef up their servers to handle other types of compression. I think .gz will be the standard for BOINC for quite a while.
[Nov 23, 2013 1:08:53 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 35   Pages: 4   [ Previous Page | 1 2 3 4 ]
[ Jump to Last Post ]
Post new Thread