We appreciate everyone's patience as we investigated this issue with the MIP research team. We'll keep you posted if a solution is found to improve the caching behavior.



Oops, thanks for pointing that out - this has now been fixed. Doug is a scientist at the Flatiron Institute and a member of the MIP research team.

Thanks everyone.

Thanks for the info doug.renfrew!

I played with limiting the number of MIP units crunched simultaniously with app_config.xml and got the following results on my two machines:

Maximum number of MIP units beyond which a noticable slowdown occurs:

2 units on i7-3770 (4 cores, 8 threads, 8 MB L3 cache)
4 units on Ryzen 1700 (8 cores, 16 threads, 16 MB L3 cache)

Other units crunched at the same time were mostly OET.

Hi doug.renfrew,
I thank you warmly for your feedback.
You confirm one of my first assumption at the launched of MIP1 regarding possible cache page fault.
I wish you and the team a lot of courage and success for trying to optimize Rosetta.
Cheers,
Yves

Points are not my primary concern, only as an indicator how effective the science is done. And as it is true, that running more MIP units always gets more work done for MIP despite a drop im efficiency, I think for WCG as a whole it is best to run a limited number of MIP units together with other projects. This should maximize the work done overall (as well as points, as an indicator of that).

And just activating all projects seems to be the easiest way to ensure this. The problem might only exist for a limited number of more active crunchers, who are more likely to activate just a few projects and get a higher proportion of MIP units therefore. It would be cool, if the problems would be solved, but to be honest, for WCG overall, the problem might be less severe than this thread indicates.

I completely agree with Apis Tintinnambulator.
On my side, I would be really happy if IT would become "greener", i.e. I mean that IT people should learn to use computational resources efficiently.
Many people and governments are "shocked" regarding the consumption of fossil energy but at the same time consider fully normal:
1/ to prefer streaming instead of download (local storage) and local re-use
2/ that bad written software and, in particular, operating systems are more and more demanding in terms of RAM, CPU performance, storage size (e.g. PC with W*, mobile phone, ...)
3/ that "modern" implies more resources demanding
4/ that "cloud computing is so cool" even cloud computing is so demanding in terms of network resources.

I am pretty convinced that spending a little bit more time for elaborating an efficient design and for performing design review can have a huge impact on the level of energy efficiency and computing effectiveness.
Since I do not compete against anybody, points are not important for me, excepted that they are supposed to reflect in some ways the computational efficiency. As soon as I discovered the efficiency issue for MIP1, I reported it because I think that, even if the contribution is "for free" from a scientist point of view, the project should handle the member computational resources in a fair and efficient way.
Happy crunching,
Yves
---
PS: My dream / hope is that, one day, Operating Systems would receive a energy efficiency label - like fridges, washing machines, TV, cars, light bulbs, etc. - and we will be able to notice that some bad written software will generate much more CO2 than some sport cars:
A - eLinux, QNX, Raspbian, MeeGo
B - Linux, Sailfish OS
C -
D -
E -
F - Windows 7, Android
G - Windows 10
(since I do not have any experience with Apple products, I am not able to rate MacOS and iOS).