I got zero units on May 17, but now have gotten 4 on May 18. For a while the units were taking about 25 hours on one machine, but the last few have been around 17 hours. I wonder if something has changed, because the machine is the same.

Cheers

No, I don't think so, because I increased the number on that machine from 4 to 5.

Cheers

Wow, Unixchick -- those run-times are impressive, even the longer ones, which are a little bit quicker than my fastest Ryzen is managing (although it's not one of those monster 150W+ CPUs which could probably shave up to an hour off if I could afford the power budget...!)

As for the runtime variations mentioned in the last few posts -- yes, there will be differences from cell to cell and from model generation to generation as conditions change, but I suspect that a "perfect" machine with a "perfect" workload would not see huge variations unless it got a short time step task or a 32-bit task. If I have a consistent workload (which doesn't happen very often at present) I see variations of up to 10% or so.

However, uneven workloads can affect run-times quite significantly. Weather models are quite hard on memory, and ARP1 has quite high levels of cache misses, even on systems with lots of L3 cache. And even something like MCM1 (which has a much smaller footprint) will slow down if there's enough going on...

If the number of MCM1 tasks running falls off because of one of those hours-long shortages of work, my ARP1 tasks generally run 5% to 10% faster (even if I have my maximum allowed number running) as there is slightly less memory contention (my systems are set up to avoid throttling because of power limits, so I know it's [mostly] memory access). Likewise, if the number of concurrent ARP1 tasks drops the MCM1 tasks run about 10% faster and any ARP1 tasks that are running will be quicker too.

If one has a server-grade motherboard and CPU(s) the effects are likely to be less obvious as there are probably more memory channels available. Also, if a machine has a reasonable amount of L3 cache (or other high-speed access to RAM?) the effect isn't as marked as it is on (say) my old i7-7700, where running more than one ARP1 at a time is pointless if anything else is going on! But there is definitely a point on any system at which running more ARP1 tasks at once is a way of earning badges rather than returning science results :-)

Cheers - Al.

P.S. -- Does anyone remember MIP1 and the machines that ran cooler the more tasks were loaded at the same time? -- that was a really bad example of a memory-bound application :-)

Great post- completely agree with what you said.

What we have seen is that memory speed is "king" for ARP1, but there are many compounding factors:

- To play off of what you said, our Threadripper Pros (5965WX (128 MB cache, 24 cores, 48 threads, runs about 4.12GHz when running all-core) will take about 6ish hours. That is 8 memory channels but we are running MCM1 work at the same time, so take that with a grain of salt. Lots of memory channels, not the fastest memory, but a fast CPU.

- Threadripper 2970WX (almost the same clock speed at the 5965WX mentioned above) will take about 13 hours with 4 memory channels but there is a significant flaw in the memory system of these CPUs.

- CPU: Intel i9 14900KS (a constantly loaded core runs at ~5.9 GHz, RAM: KLEVV CRAS V 64GB 8400MHz (out of the box rated speed). We only use a few cores of this system at the same time (2 for GPU work) and then we were running one task of ARP1 to see how it would handle it. Runtime was less than 4 hours for most ARP1 work BUT this system was seeing "invalids" for ARP1 which were not showing up on any of the other systems. We sometimes run an ARP1 on the system after we make any changes (such as bios updates) for comparative purposes.

- Old Xeons (Dell Precision)- 24 hours (those systems are rock solid, just slow..)

System RAM speed seems to make a HUGE difference for ARP1 while channels can definitely help with running a variety of work at the same time. Does anyone here have a Ryzen with the 3D stacked v-cache? I am really curious to see how those do because of the massive cache.

Sgt. Joe

A difference between 17 hours and 25 hours could correspond to a TimeStep change between 36 secs and 24 secs, but would seem to be the wrong way round.

Mike