Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 
World Community Grid Forums
  Category: Active Research
  Forum: OpenPandemics - COVID-19 Project
  Thread: How to tell job count before/during execution

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 8
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 2548 times and has 7 replies Next Thread
PMH_UK
Veteran Cruncher
UK
Joined: Apr 26, 2007
Post Count: 771
Status: Recently Active
Project Badges:
Reply to this Post  Reply with Quote 
How to tell job count before/during execution
Where can I see how many jobs are in a workunit before or during execution?
My slow Intel GPUs have some long running tasks so I would like to see how many jobs in them.
A quick search of forums and peek in files have not revealed so far.

Paul.
----------------------------------------
Paul.
[May 22, 2021 8:06:49 PM]   Link   Report threatening or abusive post: please login first  Go to top 
ca05065
Senior Cruncher
Joined: Dec 4, 2007
Post Count: 325
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: How to tell job count before/during execution
In the client_state.xml search for wcgdpf and I believe the next digit is the number of sub jobs:
-jobs OPN1_0049184_03138.job -input OPN1_0049184_03138.zip -seed 1981404240 -wcgruns 138 -wcgdpf 4

From Reults Status:
Result Name: OPN1_ 0049184_ 03138_ 0--
.
INFO:[22:27:05] Start AutoDock for ZINC001073859050_1_RX1--fr2325benz_002--CYS142_wcgsplit2.dpf(Job #0)..
.
INFO:[22:56:31] Start AutoDock for ZINC001226472181_1_RX1--fr2325benz_002--CYS142.dpf(Job #1)...
.
INFO:[23:19:53] Start AutoDock for ZINC001079553959_2_RX1--fr2325benz_002--CYS142.dpf(Job #2)...
.
INFO:[23:28:57] Start AutoDock for ZINC000854742861_1_RX1--fr2325benz_002--CYS142.dpf(Job #3)...

I realise that the above is for opn1 but I do not have any opng on my PC at the moment.
[May 22, 2021 11:16:04 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Crystal Pellet
Veteran Cruncher
Joined: May 21, 2008
Post Count: 1322
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: How to tell job count before/during execution
I'm using a Windows Powershell script for OPNG's to find the number of jobs and how far the progress is.

It's a pity, that I can't post it here because I get "Forbidden to post" when I try to Preview or Post it.
[May 23, 2021 7:34:16 AM]   Link   Report threatening or abusive post: please login first  Go to top 
squid
Advanced Cruncher
Germany
Joined: May 15, 2020
Post Count: 56
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: How to tell job count before/during execution
@PMH_UK
If you really want to crunch with an Intel GPU, wait for an Intel Xe DG2 graphics card.

https://www.pcgamer.com/intel-dg2-gaming-grap...price-performance-xe-hpg/
[May 23, 2021 8:03:37 AM]   Link   Report threatening or abusive post: please login first  Go to top 
PMH_UK
Veteran Cruncher
UK
Joined: Apr 26, 2007
Post Count: 771
Status: Recently Active
Project Badges:
Reply to this Post  Reply with Quote 
Re: How to tell job count before/during execution
@ca05065
Thanks, looks right to me.

@Crystal_Pellet
Thanks, I only have Linux on OPNG currently so will work on a script.

@squid
I am running what I have, oldish Lenovo E50-00, Intel NUC, Laptop.

Paul.
----------------------------------------
Paul.
[May 23, 2021 9:13:55 AM]   Link   Report threatening or abusive post: please login first  Go to top 
adriverhoef
Master Cruncher
The Netherlands
Joined: Apr 3, 2009
Post Count: 2167
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: How to tell job count before/during execution
Where can I see how many jobs are in a workunit before or during execution?
Paul,

To see the number of jobs in all available OPNG tasks, running or not, this could be a fine one:
  • sed -n 's/^-jobs \(OPNG_[^ ]*\)\.job .* -wcgdpf/\1, Jobs:/p' ~boinc/client_state.xml | pr -t -2
    OUTPUT (SAMPLE):
    OPNG_0010926_00195, Jobs: 16	    OPNG_0010566_00014, Jobs: 15
    OPNG_0010704_00143, Jobs: 14	    OPNG_0010566_00510, Jobs: 17
    OPNG_0011154_00071, Jobs: 46	    OPNG_0009521_00186, Jobs: 20
    OPNG_0011154_00173, Jobs: 25	    OPNG_0009521_00023, Jobs: 25
    OPNG_0010589_00019, Jobs: 20	    OPNG_0010566_00417, Jobs: 17
    OPNG_0010589_00404, Jobs: 16	    OPNG_0010566_00564, Jobs: 15
    OPNG_0011215_00160, Jobs: 39	    OPNG_0009521_00130, Jobs: 21
    OPNG_0011215_00267, Jobs: 26	    OPNG_0010566_00393, Jobs: 18
    OPNG_0011215_00146, Jobs: 46	    OPNG_0010566_00445, Jobs: 18
    OPNG_0011215_00175, Jobs: 47	    OPNG_0010843_00391, Jobs: 25
    OPNG_0011215_00159, Jobs: 37	    OPNG_0010843_00389, Jobs: 18
    OPNG_0011215_00186, Jobs: 33	    OPNG_0010843_00178, Jobs: 17
    OPNG_0011215_00165, Jobs: 28	    OPNG_0010843_00511, Jobs: 19
    OPNG_0011215_00157, Jobs: 44	    OPNG_0010843_00498, Jobs: 14
    OPNG_0011215_00050, Jobs: 33	    OPNG_0010843_00285, Jobs: 16
    OPNG_0011215_00062, Jobs: 33	    OPNG_0010843_00509, Jobs: 14
    OPNG_0011215_00158, Jobs: 31	    OPNG_0010843_00188, Jobs: 19
    OPNG_0010904_00039, Jobs: 23	    OPNG_0010324_00324, Jobs: 22
    OPNG_0009874_00381, Jobs: 19	    OPNG_0009534_00226, Jobs: 23
    OPNG_0009874_00477, Jobs: 21	    OPNG_0009597_00086, Jobs: 13
    OPNG_0009874_00494, Jobs: 13	    OPNG_0009597_00203, Jobs: 18
    OPNG_0009874_00475, Jobs: 15	    OPNG_0009610_00303, Jobs: 26
    OPNG_0009874_00070, Jobs: 18	    OPNG_0009597_00209, Jobs: 14
    OPNG_0009874_00486, Jobs: 21	    OPNG_0009610_00190, Jobs: 21
    OPNG_0009874_00107, Jobs: 20	    OPNG_0009610_00195, Jobs: 14
    OPNG_0009874_00310, Jobs: 16	    OPNG_0009922_00098, Jobs: 37
    OPNG_0009874_00537, Jobs: 18	    OPNG_0011021_00053, Jobs: 13
    OPNG_0009874_00535, Jobs: 16	    OPNG_0011021_00654, Jobs: 13
    OPNG_0009874_00369, Jobs: 17	    OPNG_0010704_00203, Jobs: 26

    To view the progress of all running tasks …
    - If there's always only one task executing, I'd use:
  • wcgresults -HN1O | sed -n 's/OPNG_.*//p' | while read slot; do [ X$slot = X- ] && continue;
    printf "Slot %d, " $slot; perl -wne 'do { s/.* -wcgdpf/Jobs:/; print } if $. == 1;
    do { ($time,$job) = /^INFO:(\[..:..:..\]).*\((Job #[^\)]+)\)/;
    print "$time $job\n" } if / Start AutoDock for /' ~boinc/slots/$slot/stderr.txt; done
    OUTPUT (SAMPLE):
    Slot 0, Jobs: 14 
    [12:48:42] Job #0
    [12:49:05] Job #1
    [12:49:32] Job #2
    [12:50:21] Job #3
    [12:51:31] Job #4
    [12:52:23] Job #5
    [12:53:09] Job #6
    [12:54:25] Job #7
    [12:54:53] Job #8
    [12:55:28] Job #9
    [12:56:00] Job #10
    [12:56:49] Job #11

    - If there is more than one task, this would be a good candidate:
  • wcgresults -HN1O | sed -n 's/OPNG_.*//p' | while read slot; do [ X$slot = X- ] && continue;
    printf "Slot %d, " $slot; perl -wne 'do { ($n) = / -wcgdpf ([^ ]*)/; print "Jobs: $n" } if $. == 1;
    ($time,$job) = /^INFO:(\[..:..:..\]).*\((Job #[^\)]+)\)/ if / Start AutoDock for /;
    BEGIN { $time = "-"; $job = "-" } END { print ", $time $job\n" }' ~boinc/slots/$slot/stderr.txt; done
    OUTPUT (SAMPLE):
    Slot 1, Jobs: 18, [13:28:56] Job #9
    Slot 0, Jobs: 19, [13:30:27] Job #11
    (EDIT) Forgot to add this: YMMV biggrin
    • ----------------------------------------
    [Edit 2 times, last edit by adriverhoef at May 23, 2021 3:16:19 PM]
    [May 23, 2021 12:23:47 PM]   Link   Report threatening or abusive post: please login first  Go to top 
    PMH_UK
    Veteran Cruncher
    UK
    Joined: Apr 26, 2007
    Post Count: 771
    Status: Recently Active
    Project Badges:
    Reply to this Post  Reply with Quote 
    Re: How to tell job count before/during execution
    Thanks for that, first works great, will try rest later.
    Linux is new-ish to me, I'm more used to things like REXX on mainframe.
    May take me a while to understand your wonderful creation.

    Paul.
    ----------------------------------------
    Paul.
    [May 23, 2021 2:53:46 PM]   Link   Report threatening or abusive post: please login first  Go to top 
    adriverhoef
    Master Cruncher
    The Netherlands
    Joined: Apr 3, 2009
    Post Count: 2167
    Status: Offline
    Project Badges:
    Reply to this Post  Reply with Quote 
    Re: How to tell job count before/during execution
    Thanks for your feedback, Paul. My knowledge of REXX is almost zero (just an hour ago it was nil, I have been looking at an introduction to REXX for a little while wink ).

    There is a small improvement in my last shell command that I would like to introduce. The output would then also contain the name of the task, like this:
    Task OPNG_0011215_00165_0, Slot 0, Jobs: 28, [18:29:13] Job #12

    The command to accomplish this would be:
  • wcgresults -HN1O | grep OPNG_ | while read slot task; do [ X$slot = X- ] && continue;
    printf "Task %s, Slot %d, " $task $slot; perl -wne 'do { ($n) = / -wcgdpf ([^ ]*)/; print "Jobs: $n" }
    if $. == 1; ($time,$job) = /^INFO:(\[..:..:..\]).*\((Job #[^\)]+)\)/ if / Start AutoDock for /;
    BEGIN { $time = "-"; $job = "-" } END { print ", $time $job\n" }' ~boinc/slots/$slot/stderr.txt; done


    EDIT: It is, of course, also possible that a resent task requests priority to be executed. In that case it's possible that one GPU task will be paused and the other one will be running instead, so that you will see output like this:
    Task OPNG_0010904_00039_0, Slot 0, Jobs: 23, [23:12:21] Job #18
    Task OPNG_0041840_00057_2, Slot 5, Jobs: 76, [01:24:57] Job #36

    Adri
    • ----------------------------------------
    [Edit 1 times, last edit by adriverhoef at May 23, 2021 11:40:30 PM]
    [May 23, 2021 4:45:43 PM]   Link   Report threatening or abusive post: please login first  Go to top 
    [ Jump to Last Post ]
    		Post new Thread