Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 4
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 2134 times and has 3 replies Next Thread
DCS1955
Veteran Cruncher
USA
Joined: May 24, 2016
Post Count: 668
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
MCM download efficiency

Does anyone know how MCM achieves such a high-efficiency of 893bytes/ work unit!!!!? Compared to other projects the download size is minuscule. I forget the specific project but some can be 200+Kbytes/WU. Is it because it has been around a while and developers have refined their design?
----------------------------------------



[May 5, 2019 1:40:07 AM]   Link   Report threatening or abusive post: please login first  Go to top 
alanb1951
Veteran Cruncher
Joined: Jan 20, 2006
Post Count: 1322
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: MCM download efficiency

Data files, like application binaries, graphics files and so on, are only downloaded if there isn't already a matching version on the client machine. So the fewer variable parts there are for a given type of project, the less persistent downloading...

There's a very large database file which is searched by MCM tasks - that only needs to be downloaded when a new data set is being scanned, so all that usually has to be downloaded for individual tasks is the "Settings File" (which isn't very big at all.)

Some projects (such as MCM) do a very large number of runs against the same data file(s), whilst others may have distinct data files for each batch (MIP1, for instance) so an individual user may only see one or two tasks that use a particular file -- more downloads, but often of much smaller files!

And, finally, some projects may have tasks that depend on the output of previous processing on the same core data (e.g. FAH2) so there are unavoidable single-task data files which might be quite large.

So it might just be that it looks more efficient, but if you were to consider the number of tasks run against a given data file (or files) the amounts of downloading might not be as disparate as they seem to be on a quick glance!

By the way, I am only an end user, so the above is based on observations and exploring the BOINC client_state.xml file and directories...

Happy crunching - Al.

[Edited to add specific reference to FAH2.]
----------------------------------------
[Edit 1 times, last edit by alanb1951 at May 5, 2019 3:09:48 AM]
[May 5, 2019 2:45:54 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: MCM download efficiency

Spot on, an initial MIP task comes with a data reference table. All you get when asking for this project's work is a pointer which from that table has to still to computed, until all for that cluster are exhausted and then a big download will come with a new set. Very efficient still.

Edit: There are/were several projects that use this technique.
----------------------------------------
[Edit 1 times, last edit by Former Member at May 5, 2019 7:48:52 AM]
[May 5, 2019 7:47:16 AM]   Link   Report threatening or abusive post: please login first  Go to top 
DCS1955
Veteran Cruncher
USA
Joined: May 24, 2016
Post Count: 668
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: MCM download efficiency

TY. Al and lava. Very interesting and informative
----------------------------------------



[May 5, 2019 2:17:11 PM]   Link   Report threatening or abusive post: please login first  Go to top 
[ Jump to Last Post ]
Post new Thread