Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 
World Community Grid Forums
  Category: Active Research
  Forum: Mapping Cancer Markers Forum
  Thread: ~35% sys calls, ok?

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 33
Posts: 33   Pages: 4   [ Previous Page | 1 2 3 4 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 7258 times and has 32 replies Next Thread
Speedy51
Veteran Cruncher
New Zealand
Joined: Nov 4, 2005
Post Count: 1326
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: ~35% sys calls, ok?
I agree with you if you are wanting to return results as fast as possible. My thoughts are with the limited amount of work there seems to be at the moment the quicker we return it the quicker we may run out of work


It is not a question of returning results as soon as possible, but of trying to run the work units where they are the most efficient. I would like to deploy my electricity where it does the most work per watt.

Cheers

+1

From my point of view you get more work per watt from Windows for SCC and more work from MCM using Linux because those other platforms that the project take longest to run on.
We are both right with what we are saying. You are not going to change my view and I am not going to change your view. :-)
Linux is not for everyone. I am guessing no one is running a Ryzen 9 7900 X if you are concerned about watts
out of interest what do you do with your watts when you run out of work. I either wait until work starts coming in all go to another project
----------------------------------------

----------------------------------------
[Edit 2 times, last edit by Speedy51 at Aug 17, 2023 4:38:25 AM]
[Aug 17, 2023 4:16:33 AM]   Link   Report threatening or abusive post: please login first  Go to top 
MJH333
Senior Cruncher
England
Joined: Apr 3, 2021
Post Count: 301
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: ~35% sys calls, ok?
AgrFan
Right now, it's best to run MCM on Windows and SCC on Linux using separate device profiles. SCC uses VINA which runs quicker on Linux than Windows.
Thank you for reminding me about this. I’ve recently been running SCC on my Windows 10 Zen 3 machine and decided to try running it on Linux instead. After quite a bit of failing to get it to work (!), I’ve now managed to get SCC to run in Ubuntu using WSL2. The results are astonishing. In Windows an SCC task would generally take 2.5 to 3 hours (with some outliers). Running on the same machine in WSL2, the tasks are generally taking about 45 minutes. Wow!
Cheers,
Mark
[Aug 17, 2023 8:19:31 AM]   Link   Report threatening or abusive post: please login first  Go to top 
BobbyB
Veteran Cruncher
Canada
Joined: Apr 25, 2020
Post Count: 638
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: ~35% sys calls, ok?
Yes a good reminder.

When WCG started having problems a week or two ago, I removed the limits for MCM on my Ubuntu's and took what there was. Have now reset them to what they were. I will keep my eye on them.

6 hours later: Yeah they are doing it OK.

I only have two Win 4core laptops so 1 SCC

But tomorrow is the start of the weekend and we know what that means.

what do you do with your watts when you run out of work. I either wait until work starts coming in all go to another project
Same here and yes am not too concerned with watts. Where I am all watts are 99.9% hydro.
----------------------------------------
[Edit 4 times, last edit by BobbyB at Aug 17, 2023 10:41:51 PM]
[Aug 17, 2023 5:11:08 PM]   Link   Report threatening or abusive post: please login first  Go to top 
cubes
World Community Grid Tech, Mapping Cancer Markers and Help Conquer Scientist
Canada
Joined: Mar 3, 2007
Post Count: 58
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: ~35% sys calls, ok?
Interesting finding -- thank you for sharing it.

For all MCM workunits these days, the bulk of the compute time is spent inside calls to the SvmLightLib API, which wraps SVM-Light. Multiple points in the SVM-Light code call the C standard library function clock(), which presumably ends up as times() system calls.

SVM training and testing is an iterative process, and different workunits will vary in the number of signatures evaluated, the number of iterations per model, and the time required per iteration, so the number of system calls and relative time spent in them will vary. The numbers are generally higher for the current dataset, but 35% does seem especially high.

Note: MCM is a unique use-case of SvmLightLib. Any problems with our application of SVM-Light/SvmLightLib to MCM are our problems, not those of the packages' authors.

I recall the WCGrid technical team mentioning paying some attention to those clock calls and trying to reduce their number in the MCM application, back when it was being adapted to WCGrid. I was strictly on the Science side of the wall back then, so couldn't see any details. No longer the case now. I'll open a ticket about the issue and look into it when time allows.
[Aug 17, 2023 5:24:18 PM]   Link   Report threatening or abusive post: please login first  Go to top 
AgrFan
Senior Cruncher
USA
Joined: Apr 17, 2008
Post Count: 396
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: ~35% sys calls, ok?
I recall the WCGrid technical team mentioning paying some attention to those clock calls and trying to reduce their number in the MCM application, back when it was being adapted to WCGrid. I was strictly on the Science side of the wall back then, so couldn't see any details. No longer the case now. I'll open a ticket about the issue and look into it when time allows.

cubes,

Thank you for taking the time to respond. This is great news. I've stopped running MCM on Ubuntu due to the long-elapsed times vs. Windows. You may want to focus on the NFCV units first since they are egregiously slow on Linux. Please keep us posted on any further developments. Thanks again.

Agrfan
----------------------------------------

  • i5-10400 (Comet Lake, 6C/12T) @ 2.9 GHz
  • i5-7400 (Kaby Lake, 4C/4T) @ 3.0 GHz
  • i5-4590 (Haswell, 4C/4T) @ 3.3 GHz
  • i5-3330 (Ivy Bridge, 4C/4T) @ 3.0 GHz

----------------------------------------
[Edit 8 times, last edit by AgrFan at Aug 17, 2023 8:24:04 PM]
[Aug 17, 2023 5:35:11 PM]   Link   Report threatening or abusive post: please login first  Go to top 
thunder7
Senior Cruncher
Netherlands
Joined: Mar 6, 2013
Post Count: 238
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: ~35% sys calls, ok?
Interesting finding -- thank you for sharing it.

...

I recall the WCGrid technical team mentioning paying some attention to those clock calls and trying to reduce their number in the MCM application, back when it was being adapted to WCGrid. I was strictly on the Science side of the wall back then, so couldn't see any details. No longer the case now. I'll open a ticket about the issue and look into it when time allows.


For the record, this is what I see on a MCM task on linux (currently @ 26.5% system time, while running a MCM and SCC mix):

sudo perf record -p <pid>
wait 10 seconds
^C
sudo perf report

Samples: 67K of event 'cycles', Event count (approx.): 46184408072                                                                                                        
Overhead  Command          Shared Object                             Symbol                                                                                               
  21.58%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] do_syscall_64                                                                                   ◆
   7.16%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] syscall_return_via_sysret                                                                       ▒
   6.34%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] entry_SYSCALL_64                                                                                ▒
   4.87%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] cputime_adjust                                                                                  ▒
   1.85%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] update_curr                                                                                     ▒
   1.79%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] thread_group_cputime                                                                            ▒
   1.78%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] entry_SYSCALL_64_after_hwframe                                                                  ▒
   1.60%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] __calc_delta                                                                                    ▒
   1.44%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] task_sched_runtime                                                                              ▒
   1.39%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] do_sys_times                                                                                    ▒
   1.17%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] __x64_sys_times                                                                                 ▒
   1.15%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] copy_user_generic_unrolled                                                                      ▒
   1.12%  wcgrid_mcm1_map  wcgrid_mcm1_map_7.61_x86_64-pc-linux-gnu  [.] 0x00000000001a6e57                                                                              ▒
   1.01%  wcgrid_mcm1_map  wcgrid_mcm1_map_7.61_x86_64-pc-linux-gnu  [.] 0x0000000000023060                                                                              ▒
   0.97%  wcgrid_mcm1_map  wcgrid_mcm1_map_7.61_x86_64-pc-linux-gnu  [.] 0x0000000000023064                                                                              ▒
   0.94%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] thread_group_cputime_adjusted                                                                   ▒
   0.93%  wcgrid_mcm1_map  wcgrid_mcm1_map_7.61_x86_64-pc-linux-gnu  [.] 0x0000000000023021                                                                              ▒
   0.92%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] __lock_text_start                                                                               ▒
   0.89%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] _raw_spin_lock_irqsave                                                                          ▒
   0.86%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] nsec_to_clock_t                                                                                 ▒
   0.80%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] update_rq_clock                                                                                 ▒
   0.78%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] cpuacct_charge                                                                                  ▒
   0.75%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] task_rq_lock                                                                                    ▒
   0.74%  wcgrid_mcm1_map  [kernel.kallsyms]                         [k] native_sched_clock                                                                              ▒
   0.73%  wcgrid_mcm1_map  wcgrid_mcm1_map_7.61_x86_64-pc-linux-gnu  [.] 0x0000000000022c44                                                                              ▒
   0.61%  wcgrid_mcm1_map  wcgrid_mcm1_map_7.61_x86_64-pc-linux-gnu  [.] 0x000000000002301c                                                                              ▒
   0.55%  wcgrid_mcm1_map  wcgrid_mcm1_map_7.61_x86_64-pc-linux-gnu  [.] 0x000000000016b48e                                                                              ▒
   0.54%  wcgrid_mcm1_map  wcgrid_mcm1_map_7.61_x86_64-pc-linux-gnu  [.] 0x0000000000022c4e                                                                              ▒
   0.54%  wcgrid_mcm1_map  wcgrid_mcm1_map_7.61_x86_64-pc-linux-gnu  [.] 0x0000000000022c4c
▒
0.51% wcgrid_mcm1_map [kernel.kallsyms] [k] update_min_vruntime
----------------------------------------
[Edit 1 times, last edit by thunder7 at Aug 17, 2023 7:32:41 PM]
[Aug 17, 2023 7:32:02 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sgt.Joe
Ace Cruncher
USA
Joined: Jul 4, 2006
Post Count: 7846
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: ~35% sys calls, ok?
out of interest what do you do with your watts when you run out of work

When I run out of work and the backup project has no work, I turn off the machines. When they are off they use zero watts.
Cheers
----------------------------------------
Sgt. Joe
*Minnesota Crunchers*
[Aug 18, 2023 1:52:05 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Speedy51
Veteran Cruncher
New Zealand
Joined: Nov 4, 2005
Post Count: 1326
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: ~35% sys calls, ok?
out of interest what do you do with your watts when you run out of work

When I run out of work and the backup project has no work, I turn off the machines. When they are off they use zero watts.
Cheers

The only way I am aware of to know they will use 0 watts for sure is if you unplug PC/screen from the wall.
----------------------------------------
[Aug 18, 2023 3:21:58 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Sgt.Joe
Ace Cruncher
USA
Joined: Jul 4, 2006
Post Count: 7846
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: ~35% sys calls, ok?
out of interest what do you do with your watts when you run out of work

When I run out of work and the backup project has no work, I turn off the machines. When they are off they use zero watts.
Cheers

The only way I am aware of to know they will use 0 watts for sure is if you unplug PC/screen from the wall.

If I have them turned off, I unplug them. If they are off, there is no need to have them plugged in.
Cheers
----------------------------------------
Sgt. Joe
*Minnesota Crunchers*
[Aug 18, 2023 3:23:13 PM]   Link   Report threatening or abusive post: please login first  Go to top 
BobbyB
Veteran Cruncher
Canada
Joined: Apr 25, 2020
Post Count: 638
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: ~35% sys calls, ok?
Is there really anyone here worried about watts except as discussed above. MCM better on Windows and SCC better on Linux. smile

I appreciate that co$t is a factor in some places and I bet in these cases they have cut back on time and/or machines.

If watts is THE factor then you are probably not doing this anymore and are not reading this post. The fact we are here indicates we made the conscious decision that the project is worth it along with the backup projects also.
[Aug 18, 2023 4:56:19 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 33   Pages: 4   [ Previous Page | 1 2 3 4 | Next Page ]
[ Jump to Last Post ]
Post new Thread