Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 65
Posts: 65   Pages: 7   [ Previous Page | 1 2 3 4 5 6 7 ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 13419 times and has 64 replies Next Thread
adriverhoef
Master Cruncher
The Netherlands
Joined: Apr 3, 2009
Post Count: 2346
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Version 1.34.7 of my script 'wcgresults'

You may find the current version here.
What's new in version 1.34.7 of 'wcgresults' since version 1.30.1?

- Option -H has been added to omit the first output line when using option -r etc.
Adding another -H will revert the effect and only show the first line of output.
Examples:
$ wcgresults -Hz
MIP1_00004124_1084_0(S)
$ wcgresults -HHr
91 tasks: 84 downloaded, 7 uploaded
- Option -M has been added to show all tasknames on the device in order of receipt.
- Option -q has been changed, this will now also show the number of devices with tasks.
Example:
$ wcgresults -dqq
20:30:00 a_device scc1:39 android_device zika:2 b_device mip1:9 scc1:23 = 73 @ 3
("@ 3" denotes a total of 3 devices, "= 73" denotes a total of 73 tasks)
- Option -Q has been added to show the number of tasks and their statuses per project.
Example:
$ wcgresults -Q
42 tasks: 39 downloaded, 2 uploaded, 1 uploading
MCM1 (#):34 [(0):12 (1):18 (2):3 (3):1] (R):4 (U):1 (.):2
MIP1 (S):1
(34 MCM1-tasks are divided into 12x "_0", 18x "_1", 3x "_2" and one "_3".)
- just a few bugfixes, e.g. the situation where the API doesn't return anything at all.

----------------------------------------
[Edit 3 times, last edit by adriverhoef at Sep 15, 2017 7:21:59 PM]
[Sep 15, 2017 6:38:57 PM]   Link   Report threatening or abusive post: please login first  Go to top 
adriverhoef
Master Cruncher
The Netherlands
Joined: Apr 3, 2009
Post Count: 2346
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Patching version 1.30.1 of my script 'wcgresults' to 1.34.7

To patch version 1.30.1 of my script 'wcgresults' to 1.34.7 AND keeping your configuration, you may want to save the code below to 'patchfile' (by clicking "Quote" and selecting the code), make a safety copy of your configured 1.30.1 version, make sure the file 'wcgresults' is in the current directory and then run patch -p0 <patchfile.
*** wcgresults	2017-09-15 16:10:28.620192878 +0200
--- wcgresults.old 2017-09-15 16:11:44.182919657 +0200
***************
*** 1,9 ****
! #!/bin/sh
#
# wcgresults - use WCG's API to get a list of recent results
# https://www.worldcommunitygrid.org/help/viewTopic.do?shortName=api
#
! # VERSION: 1.30.1
# Author: Adri Verhoef

Prog=${0##*/}
--- 1,9 ----
! #!/bin/bash
#
# wcgresults - use WCG's API to get a list of recent results
# https://www.worldcommunitygrid.org/help/viewTopic.do?shortName=api
#
! # VERSION: 1.34.7
# Author: Adri Verhoef

Prog=${0##*/}
***************
*** 26,32 ****
printf "%s\n" \
"Usage: $Prog [options] [-d | file ...]" \
" Options:" \
! " -A omit ANSI-code [can be used with options -q, -r and -t]" \
" -a [see option -f] also record results that got 0 points all in all" \
" -b sortby sort results by the value of ’sortby’" \
" (0 = DeviceId, 1 = SentTime, 2 = ReportDeadline, 3 = ReceivedTime (default), 4 = CpuTime)" \
--- 26,32 ----
printf "%s\n" \
"Usage: $Prog [options] [-d | file ...]" \
" Options:" \
! " -A omit ANSI-code [can be used with options -q, -r, -t and -z]" \
" -a [see option -f] also record results that got 0 points all in all" \
" -b sortby sort results by the value of ’sortby’" \
" (0 = DeviceId, 1 = SentTime, 2 = ReportDeadline, 3 = ReceivedTime (default), 4 = CpuTime)" \
***************
*** 34,62 ****
" -D dir download new results to directory ’dir’" \
" -d download new results (for all devices)" \
" -f logfile record results (for all devices) that went through the validator to file ’logfile’, sorted by CpuTime" \
" -i info depending on the value of ’info’ (E, O, S, or V)," \
" E only show the textvalue of each defined ExitStatus >= 192" \
" O only show the textvalue of each defined Outcome" \
" S only show the textvalue of each defined ServerState" \
" V only show the textvalue of each defined ValidateState" \
" -l limit define the number of results to download (default: 250)" \
! " -m show all tasknames on this device, each listed with ”report deadline” and ”fraction done”" \
" -o offset define the number of results to skip when downloading (default: 0)" \
! " -q only show the number of jobs in progress per device (”-qq” will also list the apps) on one line" \
" -r only show the names of all running tasks in the queue on this device" \
" -s state only select results with ”ValidateState” value ’state’" \
" (0 = pending validation, 1 = valid, 2 = invalid," \
" 4 = pending verification, 5 = results failed to validate within given deadline)" \
! " -t only show all tasknames in the queue on this device; use -T to show only one taskname per line" \
" -u user the name of your account on World Community Grid (default: you; ’user’ = 1 ... for other account(s))" \
" -w only select results (max. 250) (for all devices) that are still 'in progress'" \
! " -x retry all current file transfer(s) on this device" \
" -z only show the names of all suspended tasks in the queue on this device" \
;
}

# Options
! OPTS=hAab:cD:df:i:l:mo:qrs:tTu:wxz
export opt_A=0 # 1: omit ANSI
export opt_a=0
export opt_b=3 # latest received
--- 34,64 ----
" -D dir download new results to directory ’dir’" \
" -d download new results (for all devices)" \
" -f logfile record results (for all devices) that went through the validator to file ’logfile’, sorted by CpuTime" \
+ " -H omit header ”... tasks: ...” [can be used with options -r, -t and -z]; add another -H to reverse the effect" \
" -i info depending on the value of ’info’ (E, O, S, or V)," \
" E only show the textvalue of each defined ExitStatus >= 192" \
" O only show the textvalue of each defined Outcome" \
" S only show the textvalue of each defined ServerState" \
" V only show the textvalue of each defined ValidateState" \
" -l limit define the number of results to download (default: 250)" \
! " -m show all tasknames on this device, each listed with ”report deadline” and ”fraction done”; use -M to show only tasknames" \
" -o offset define the number of results to skip when downloading (default: 0)" \
! " -q only show the number of tasks in progress per device (”-qq” will also list the apps) on one line" \
! " -Q only show the number of tasks and their statuses per project on this device" \
" -r only show the names of all running tasks in the queue on this device" \
" -s state only select results with ”ValidateState” value ’state’" \
" (0 = pending validation, 1 = valid, 2 = invalid," \
" 4 = pending verification, 5 = results failed to validate within given deadline)" \
! " -t only show all tasknames in the queue on this device with their statuses; use -T to show only one taskname per line" \
" -u user the name of your account on World Community Grid (default: you; ’user’ = 1 ... for other account(s))" \
" -w only select results (max. 250) (for all devices) that are still 'in progress'" \
! " -x retry current file transfer(s) on this device" \
" -z only show the names of all suspended tasks in the queue on this device" \
;
}

# Options
! OPTS=hAab:cD:df:Hi:l:Mmo:Qqrs:Ttu:wxz
export opt_A=0 # 1: omit ANSI
export opt_a=0
export opt_b=3 # latest received
***************
*** 64,73 ****
--- 66,78 ----
export opt_D=$TMPDIR
export opt_d=0
export opt_f=""
+ export opt_H=0 # 1: omit header; >1: omit all but the header
export opt_i=""
export opt_l=250
+ export opt_M=0
export opt_m=0
export opt_o=0
+ export opt_Q=0 # 1: show the queue (the number of tasks and their statuses per project) on this device
export opt_q=0 # 1: show the queue of all devices on one line
export opt_r=0 # 1: show running tasks on this device
export opt_s=""
***************
*** 80,86 ****
while getopts $OPTS opt; do
case $opt in
h) usage; exit 0;;
! [AacdmqrTtwxz]) eval let opt_$opt++;;
[a-zA-Z0-9]) eval opt_$opt='$OPTARG';;
?) usage; exit 1;;
esac
--- 85,91 ----
while getopts $OPTS opt; do
case $opt in
h) usage; exit 0;;
! [AacdHMmQqrTtwxz]) eval let opt_$opt++;;
[a-zA-Z0-9]) eval opt_$opt='$OPTARG';;
?) usage; exit 1;;
esac
***************
*** 88,98 ****
shift $(($OPTIND - 1))

case $opt_b in [01234]) case $opt_b in 0) opt_b=DeviceId;; 1) opt_b=SentTime;; 2) opt_b=ReportDeadline;; 3) opt_b=ReceivedTime;; 4) opt_b=CpuTime;; esac; SortBy="&SortBy=$opt_b";; *) SortBy="";; esac
case $opt_o in 0) Offset="";; *) Offset="&Offset=$opt_o";; esac
case $opt_q in 0) ;; *) opt_w=1;; esac # see below
case $opt_r in 0) ;; *) opt_t=1;; esac
case $opt_s in [01245]) ValidateState="&ValidateState=$opt_s";; *) ValidateState="";; esac
! case $opt_T in 0) ;; *) opt_t=1;; esac
case $opt_u in "") ;; *) eval MEMBER=\$MEMBER$opt_u; eval VERIFY=\$VERIFY$opt_u;; esac
case $opt_w in 0) ServerState=5;; *) ServerState=4;; esac
case $opt_z in 0) ;; *) opt_t=1;; esac
--- 93,105 ----
shift $(($OPTIND - 1))

case $opt_b in [01234]) case $opt_b in 0) opt_b=DeviceId;; 1) opt_b=SentTime;; 2) opt_b=ReportDeadline;; 3) opt_b=ReceivedTime;; 4) opt_b=CpuTime;; esac; SortBy="&SortBy=$opt_b";; *) SortBy="";; esac
+ case $opt_M in 0) ;; *) opt_m=$opt_M;; esac
case $opt_o in 0) Offset="";; *) Offset="&Offset=$opt_o";; esac
+ case $opt_Q in 0) ;; *) opt_t=$opt_Q;; esac
case $opt_q in 0) ;; *) opt_w=1;; esac # see below
case $opt_r in 0) ;; *) opt_t=1;; esac
case $opt_s in [01245]) ValidateState="&ValidateState=$opt_s";; *) ValidateState="";; esac
! case $opt_T in 0) ;; *) opt_t=$opt_T;; esac
case $opt_u in "") ;; *) eval MEMBER=\$MEMBER$opt_u; eval VERIFY=\$VERIFY$opt_u;; esac
case $opt_w in 0) ServerState=5;; *) ServerState=4;; esac
case $opt_z in 0) ;; *) opt_t=1;; esac
***************
*** 128,133 ****
--- 135,142 ----
PRINT_HEADER_ONLY='
printf "\t''\t''\t''''\t'"$ADDITIONAL_HEADER"'\n"
'
+ PERL_EXPLAIN_1='(A) = Aborted; (C) = Computation error; (D) = Downloading; (N) = New; (R) = Running;'
+ PERL_EXPLAIN_2='(S) = Suspended; (U) = Uploading; (W) = Waiting to run; (.) = Ready to report'

calculate_or_record () {
perl -wne '
***************
*** 235,259 ****
}
chomp;
if (/"ResultsReturned": "([^"]+)",/) { print "Results returned: $1\n" unless $ENV{opt_f} ne ""; }
! elsif (/"AppName": "([^"]+)",/) { $AppName = $1; }
! elsif (/"ClaimedCredit": ([^,]+)/) { $ClaimedCredit = $1; }
! elsif (/"CpuTime": ([^,]+)/) { $CpuTime = $1; }
! elsif (/"ElapsedTime": ([^,]+)/) { $ElapsedTime = $1; }
! elsif (/"ExitStatus": ([^,]+)/) { $ExitStatus = $1; }
! elsif (/"GrantedCredit": ([^,]+)/) { $GrantedCredit = $1; }
! elsif (/"DeviceId": ([^,]+)/) { $DeviceId = $1; } # only used for LOGFILE
! elsif (/"DeviceName": "([^"]+)",/) { $DeviceName = $1; }
! elsif (/"ModTime": ([^,]+)/) { $ModTime = $1; } # only used for LOGFILE
! elsif (/"WorkunitId": ([^,]+)/) { $WorkunitId = $1; } # only used for LOGFILE
! elsif (/"ResultId": ([^,]+)/) { $ResultId = $1; } # only used for LOGFILE
! elsif (/"Name": "([^"]+)",/) { $Name = $1; }
! elsif (/"Outcome": ([^,]+)/) { $Outcome = $1; }
! elsif (/"ReceivedTime": "([^"]+)",/) { $ReceivedTime = $1; } # only used for LOGFILE
! elsif (/"ReportDeadline": "([^"]+)",/) { $ReportDeadline = $1; } # only used for LOGFILE
! elsif (/"SentTime": "([^"]+)",/) { $SentTime = $1; } # only used for LOGFILE
! elsif (/"ServerState": ([^,]+)/) { $ServerState = $1; }
! elsif (/"ValidateState": ([^,]+)/) { $ValidateState = $1; }
! elsif (/"FileDeleteState": ([^,]+)/) { $FileDeleteState = $1; } # only used for LOGFILE
if (/"([^"]+)": ("([^"]+)"|[0-9]+(\.[0-9]+)?)$/) { # last item in record, line not ending in comma
if ($ENV{opt_f} ne "") {
# ServerState == 5: results that have already been reported back to the server
--- 244,293 ----
}
chomp;
if (/"ResultsReturned": "([^"]+)",/) { print "Results returned: $1\n" unless $ENV{opt_f} ne ""; }
! else {
! if (/: "([^"]+)",?/) { $value = $1 }
! elsif (/: ([^,]+)/) { $value = $1 }
! # In order to initialize all fields we need to have one grip: Appname is the first field. # 09-07-2017
! if (/"AppName": /) {
! $AppName = $value;
! $ClaimedCredit = 0;
! $CpuTime = 0;
! $ElapsedTime = 0;
! $ExitStatus = 0;
! $GrantedCredit = 0;
! $DeviceId = 0;
! $DeviceName = "";
! $ModTime = 0;
! $WorkunitId = 0;
! $ResultId = 0;
! $Name = "";
! $Outcome = 0;
! $ReceivedTime = "";
! $ReportDeadline = "";
! $SentTime = "";
! $ServerState = 0;
! $ValidateState = 0;
! $FileDeleteState = 0;
! }
! elsif (/"ClaimedCredit": /) { $ClaimedCredit = $value; }
! elsif (/"CpuTime": /) { $CpuTime = $value; }
! elsif (/"ElapsedTime": /) { $ElapsedTime = $value; }
! elsif (/"ExitStatus": /) { $ExitStatus = $value; }
! elsif (/"GrantedCredit": /) { $GrantedCredit = $value; }
! elsif (/"DeviceId": /) { $DeviceId = $value; } # only used for LOGFILE
! elsif (/"DeviceName": /) { $DeviceName = $value; }
! elsif (/"ModTime": /) { $ModTime = $value; } # only used for LOGFILE
! elsif (/"WorkunitId": /) { $WorkunitId = $value; } # only used for LOGFILE
! elsif (/"ResultId": /) { $ResultId = $value; } # only used for LOGFILE
! elsif (/"Name": /) { $Name = $value; }
! elsif (/"Outcome": /) { $Outcome = $value; }
! elsif (/"ReceivedTime": /) { $ReceivedTime = $value; } # only used for LOGFILE
! elsif (/"ReportDeadline": /) { $ReportDeadline = $value; } # only used for LOGFILE
! elsif (/"SentTime": /) { $SentTime = $value; } # only used for LOGFILE
! elsif (/"ServerState": /) { $ServerState = $value; }
! elsif (/"ValidateState": /) { $ValidateState = $value; }
! elsif (/"FileDeleteState": /) { $FileDeleteState = $value; } # only used for LOGFILE
! }
if (/"([^"]+)": ("([^"]+)"|[0-9]+(\.[0-9]+)?)$/) { # last item in record, line not ending in comma
if ($ENV{opt_f} ne "") {
# ServerState == 5: results that have already been reported back to the server
***************
*** 376,421 ****
printf "Invalid time, try 1h30m0s or 50m22s or 3600s\n\n";
$secs = 0;
}
! printf "You could set your jobcache to %.2f days.\n\n", $secs * 35 / 86400 if $secs > 0;
print $ask;
'
}

do_opt_m () {
! let part=0
! TMP_BOINCCMD=/tmp/boinccmd.$$
! boinccmd --get_tasks > $TMP_BOINCCMD.$part
! for key in "report deadline" "fraction done" "name"; do
! let part++
! sed -n "s/^ $key: //p" $TMP_BOINCCMD.0 > $TMP_BOINCCMD.$part
! done
! if [ $opt_m -gt 1 ]; then
! FOURDASH=----
! DDDDDASH=$FOURDASH$FOURDASH$FOURDASH$FOURDASH
! printf "%24s\t%-8s\t%s\n" "${DDDDDASH}Deadline" "Fraction" "Name${DDDDDASH}"
fi
- # Sample output from 'paste':
- # Tue Mar 14 12:05:37 2017 0.237921 HST1_008952_000037_MC0024_T400_F00020_S00008_0
- paste $TMP_BOINCCMD.[123]
- rm -f $TMP_BOINCCMD.[0-3]
}

do_opt_t () {
if [ $opt_r = 0 -a $opt_z = 0 ]; then
# opt_t
! PERL_PUT_STATE='printf "%s(%s)%s", '"$PERL_INIT_ANSI"', $s, '"$PERL_EXIT_ANSI"''
! PERL_TERM_SIZE='
! use Term::Size "chars"; # Fedora: dnf install perl-Term-Size
! '
! PERL_INIT_COLS='
! $maxl = 0; # find longest taskname
! (($cols, $rows) = chars(STDOUT)) || ($cols = 0);
! $cols = 80 if $cols == 0;
! '
! PERL_INIT_MAXL='
! $x = length $1;
! $maxl = $x if $x > $maxl;
'
PERL_INIT_WHAT='
$what{$1} = "";
'
--- 410,472 ----
printf "Invalid time, try 1h30m0s or 50m22s or 3600s\n\n";
$secs = 0;
}
! printf "You could set your taskcache to %.2f days.\n\n", $secs * 70 / 86400 if $secs > 0; # Maximum number of WUs per core = 70 since the second half of May 2017 (old value was 35).
print $ask;
'
}

do_opt_m () {
! FOURDASH=----
! DDDDDASH=$FOURDASH$FOURDASH$FOURDASH$FOURDASH
! name="Name${DDDDDASH}"
! if [ $opt_M = 0 ]; then
! let part=0
! TMP_BOINCCMD=/tmp/boinccmd.$$
! boinccmd --get_tasks > $TMP_BOINCCMD.$part
! for key in "report deadline" "fraction done" "name"; do
! let part++
! sed -n "s/^ $key: //p" $TMP_BOINCCMD.0 > $TMP_BOINCCMD.$part
! done
! if [ $opt_m -gt 1 ]; then
! printf "%24s\t%-8s\t%s\n" "${DDDDDASH}Deadline" "Fraction" "$name"
! fi
! # Sample output from 'paste':
! # Tue Mar 14 12:05:37 2017 0.237921 HST1_008952_000037_MC0024_T400_F00020_S00008_0
! paste $TMP_BOINCCMD.[123]
! rm -f $TMP_BOINCCMD.[0-3]
! else
! if [ $opt_M -gt 1 ]; then
! printf "%s\n" "$name"
! fi
! boinccmd --get_tasks | sed -n "s/^ name: //p"
fi
}

do_opt_t () {
if [ $opt_r = 0 -a $opt_z = 0 ]; then
# opt_t
! PERL_A_C_D_N_U='
! $WHAT{"aborted"} = "A";
! $WHAT{"compute error"} = "C";
! $WHAT{"downloading"} = "D";
! $WHAT{"new"} = "N";
! $WHAT{"uploading"} = "U";
'
+ PERL_PUT_STATE='printf "%s(%s)%s", '"$PERL_INIT_ANSI"', $s, '"$PERL_EXIT_ANSI"''
+ if [ $opt_Q = 0 ]; then
+ PERL_TERM_SIZE='
+ use Term::Size "chars"; # Fedora: dnf install perl-Term-Size; Ubuntu: apt-get install libterm-size-perl
+ '
+ PERL_INIT_COLS='
+ $maxl = 0; # find longest taskname
+ (($cols, $rows) = chars(STDOUT)) || ($cols = 0);
+ $cols = 80 if $cols == 0;
+ '
+ PERL_INIT_MAXL='
+ $x = length $1;
+ $maxl = $x if $x > $maxl;
+ '
+ fi
PERL_INIT_WHAT='
$what{$1} = "";
'
***************
*** 435,497 ****
}
'
PERL_FIND_FRAC=""
! PERL_EXPLAIN_1='(A) = Aborted; (C) = Computation error; (D) = Downloading; (N) = New; (R) = Running;'
! PERL_EXPLAIN_2='(S) = Suspended; (U) = Uploading; (W) = Waiting to run; (.) = Ready to report'
! PERL_FINAL_CUT='
! do {
! if ($cols <= 80) { $sep = "\n" } else { $sep = " " }
! printf "'"$PERL_EXPLAIN_1"'%s'"$PERL_EXPLAIN_2"'\n", $sep;
! } if $ENV{opt_t} > 1;
! # Try fitting all names in 10 textcolumns first (with 2 spaces inbetween):
! $x=0;
! for ($k = 10; $k >= 2; $k--) {
! # 3 spaces for states "(.)" or "(R)", 1 for room between columns, 3 for states in last column
! if (($maxl + 3+1) * ($k-1) + $maxl + 3 < $cols) {
! $x=1; # found a suitable maximum number of textcolumns
! last;
! }
! last if $x == 1;
! }
! $k = 1 if $ENV{opt_T} > 0;
! $rows = int(($n + $k-1) / $k);
! $WHAT{"aborted"} = "A";
! $WHAT{"compute error"} = "C";
! $WHAT{"downloading"} = "D";
! $WHAT{"new"} = "N";
! $WHAT{"uploading"} = "U";
! for ($r = 1; $r <= $rows; $r++) {
! for ($j = 1; $j <= $k; $j++) {
! $seq = $r + ($j-1)*$rows;
! last if $seq > $n;
! $wu = $name{$seq};
! ($w, $u) = $wu =~ /(.*_)([0-9]+)/;
! if ($u > 0) {
! printf "%s%s%s%s", $w, '"$PERL_INIT_ANSI"', $u, '"$PERL_EXIT_ANSI"';
! } else {
! printf "%s", $wu;
}
! $x = length $name{$seq};
if ($acts{$seq} eq "EXECUTING") {
! $s = "R"; '"$PERL_PUT_STATE"';
! $x += 3;
! } elsif (exists $WHAT{$what{$wu}}) {
! printf "(%s)", $WHAT{$what{$wu}};
! $x += 3;
} elsif ($susp{$seq} eq "yes") {
! $s = "S"; '"$PERL_PUT_STATE"';
! $x += 3;
} elsif ($acts{$seq} eq "SUSPENDED") {
! printf "(W)";
! $x += 3;
} elsif ($rtor{$seq} eq "yes") {
! print "(.)";
! $x += 3;
}
! print " " x (1 + $maxl+3-$x) if $j < $k;
}
printf "\n";
! }
! '
elif [ $opt_z = 0 ]; then
# opt_r
PERL_TERM_SIZE=""
--- 486,606 ----
}
'
PERL_FIND_FRAC=""
! if [ $opt_Q = 0 ]; then
! PERL_EXPLAINER='
! do {
! if ($cols <= 80) { $sep = "\n" } else { $sep = " " }
! printf "'"$PERL_EXPLAIN_1"'%s'"$PERL_EXPLAIN_2"'\n", $sep;
! } if $ENV{opt_t} > 1;
! '
! PERL_FINAL_CUT='
! # Try fitting all names in 10 textcolumns first (with 2 spaces inbetween):
! $x=0;
! for ($k = 10; $k >= 2; $k--) {
! # 3 spaces for states "(.)" or "(R)", 1 for room between columns, 3 for states in last column
! if (($maxl + 3+1) * ($k-1) + $maxl + 3 < $cols) {
! $x=1; # found a suitable maximum number of textcolumns
! last;
! }
! last if $x == 1;
! }
! $k = 1 if $ENV{opt_T} > 0;
! $rows = int(($n + $k-1) / $k);
! '"$PERL_A_C_D_N_U"'
! for ($r = 1; $r <= $rows; $r++) {
! for ($j = 1; $j <= $k; $j++) {
! $seq = $r + ($j-1)*$rows;
! last if $seq > $n;
! $wu = $name{$seq};
! ($w, $u) = $wu =~ /(.*_)([0-9]+)/;
! if ($u > 0) {
! printf "%s%s%s%s", $w, '"$PERL_INIT_ANSI"', $u, '"$PERL_EXIT_ANSI"';
! } else {
! printf "%s", $wu;
! }
! $x = length $name{$seq};
! if ($acts{$seq} eq "EXECUTING") {
! $s = "R"; '"$PERL_PUT_STATE"';
! $x += 3;
! } elsif (exists $WHAT{$what{$wu}}) {
! printf "(%s)", $WHAT{$what{$wu}};
! $x += 3;
! } elsif ($susp{$seq} eq "yes") {
! $s = "S"; '"$PERL_PUT_STATE"';
! $x += 3;
! } elsif ($acts{$seq} eq "SUSPENDED") {
! printf "(W)";
! $x += 3;
! } elsif ($rtor{$seq} eq "yes") {
! print "(.)";
! $x += 3;
! }
! print " " x (1 + $maxl+3-$x) if $j < $k;
}
! printf "\n";
! }
! '
! else
! # opt_Q
! PERL_TERM_SIZE=""
! PERL_INIT_COLS=""
! PERL_INIT_MAXL=""
! PERL_EXPLAINER='
! do {
! if ($ENV{opt_t} > 2) { $sep = "\n" } else { $sep = " " }
! printf "'"$PERL_EXPLAIN_1"'%s'"$PERL_EXPLAIN_2"'\n", $sep;
! } if $ENV{opt_t} > 1;
! '
! PERL_FINAL_CUT='
! '"$PERL_A_C_D_N_U"'
! for ($seq=1; $seq<=$n; $seq++) {
! $task = $name{$seq};
! ($app) = $task =~ /^([^_]+)_/; # name of the app
! ($u) = $task =~ /_([0-9])$/; # task-sequence of the workunit
if ($acts{$seq} eq "EXECUTING") {
! ++$apps{$app}{"R"};
! } elsif (exists $WHAT{$what{$task}}) {
! ++$apps{$app}{$WHAT{$what{$task}}};
} elsif ($susp{$seq} eq "yes") {
! ++$apps{$app}{"S"};
} elsif ($acts{$seq} eq "SUSPENDED") {
! ++$apps{$app}{"W"};
} elsif ($rtor{$seq} eq "yes") {
! ++$apps{$app}{"."};
! } else {
! ++$apps{$app}{$u};
! ++$apps{$app}{"#"};
! }
! }
! $seq = 0;
! foreach $app (sort (keys %apps)) {
! printf "\n" if $seq > 0;
! printf "%s%s%s", '"$PERL_INIT_ANSI"', $app, '"$PERL_EXIT_ANSI"';
! foreach $X (("#", "A", "C", "D", "N", "R", "S", "U", "W", ".")) {
! do {
! if ($X eq "#") {
! if ($apps{$app}{$X} == $apps{$app}{"0"}) {
! print " (0):$apps{$app}{$X}";
! } else {
! print " (#):$apps{$app}{$X} [";
! foreach $H (("0", "1", "2", "3", "4", "5", "6", "7", "8", "9")) {
! do {
! print " " if $H ne "0";
! print "($H):$apps{$app}{$H}";
! } if exists $apps{$app}{$H};
! }
! print "]";
! }
! } else {
! print " ($X):$apps{$app}{$X}";
! }
! } if exists $apps{$app}{$X};
}
! $seq++;
}
printf "\n";
! '
! fi
elif [ $opt_z = 0 ]; then
# opt_r
PERL_TERM_SIZE=""
***************
*** 507,512 ****
--- 616,622 ----
next;
}
'
+ PERL_EXPLAINER=""
PERL_FINAL_CUT='
for ($seq = 1; $seq <= $n; $seq++) {
printf "%s%s(R)%s[%.3f%%]\n", $name{$seq}, '"$PERL_INIT_ANSI"', '"$PERL_EXIT_ANSI"', 100*$frac{$seq} if $acts{$seq} eq "EXECUTING"; # active task state
***************
*** 528,533 ****
--- 638,644 ----
}
'
PERL_FIND_FRAC=""
+ PERL_EXPLAINER=""
PERL_FINAL_CUT='
for ($seq = 1; $seq <= $n; $seq++) {
printf "%s%s(S)%s\n", $name{$seq}, '"$PERL_INIT_ANSI"', '"$PERL_EXIT_ANSI"' if $susp{$seq} eq "yes";
***************
*** 541,546 ****
--- 652,658 ----
BEGIN {
$n = 0; # count the number of tasks
'"$PERL_INIT_COLS"'
+ $running = 0; # 23-04-2017, BUGFIX: do not assume tasks are running
$suspended = 0;
}
if (/^([1-9][0-9]*)\) --+/) {
***************
*** 568,583 ****
'"$PERL_FIND_FRAC"'
END {
exit(0) if $n == 0;
! printf "$n task%s", "s" x ($n > 1);
! $sep = ":";
! foreach $state (sort keys %load) {
! printf "%s %d %s", $sep, $load{$state}, $state;
! $sep = ",";
! }
! printf "%s %d running", $sep, $running if $ENV{opt_r} > 1;
! printf "%s %d suspended", $sep, $suspended if $ENV{opt_z} > 1;
! printf "\n";
! '"$PERL_FINAL_CUT"'
}
'
}
--- 680,702 ----
'"$PERL_FIND_FRAC"'
END {
exit(0) if $n == 0;
! do {
! '"$PERL_EXPLAINER"'
! } unless $ENV{opt_H} > 1;
! do {
! printf "$n task%s", "s" x ($n > 1);
! $sep = ":";
! foreach $state (sort keys %load) {
! printf "%s %d %s", $sep, $load{$state}, $state;
! $sep = ",";
! }
! printf "%s %d running", $sep, $running if $ENV{opt_r} > 1;
! printf "%s %d suspended", $sep, $suspended if $ENV{opt_z} > 1;
! printf "\n";
! } unless $ENV{opt_H} == 1;
! do {
! '"$PERL_FINAL_CUT"'
! } unless $ENV{opt_H} > 1;
}
'
}
***************
*** 587,593 ****
'"$PERL_ANSICOLOR"'
use POSIX; # strftime()
BEGIN {
! $total = 0;
}
chomp;
@t = split(/\t/);
--- 706,713 ----
'"$PERL_ANSICOLOR"'
use POSIX; # strftime()
BEGIN {
! $tasks = 0;
! $devices = 0;
}
chomp;
@t = split(/\t/);
***************
*** 595,601 ****
# column 7: DeviceName
$app{$t[7]}{$t[0]}++;
$dev{$t[7]}++;
! $total++;
END {
if ($ENV{opt_d} == 0) {
printf "-";
--- 715,721 ----
# column 7: DeviceName
$app{$t[7]}{$t[0]}++;
$dev{$t[7]}++;
! $tasks++;
END {
if ($ENV{opt_d} == 0) {
printf "-";
***************
*** 603,608 ****
--- 723,729 ----
print strftime "%T", localtime; # HH:MM:SS
}
foreach $devkey (sort keys %dev) {
+ $devices++;
printf " %s%s%s", '"$PERL_INIT_ANSI"', $devkey, '"$PERL_EXIT_ANSI"';
if ($ENV{opt_q} == 1) {
printf " %d", $dev{$devkey};
***************
*** 612,618 ****
}
}
}
! printf " %s%s%s %d\n", '"$PERL_INIT_ANSI"', "=", '"$PERL_EXIT_ANSI"', $total;
}
' $opt_f
rm -f $opt_f
--- 733,741 ----
}
}
}
! printf " %s=%s %d", '"$PERL_INIT_ANSI"', '"$PERL_EXIT_ANSI"', $tasks;
! printf " %s@%s %d", '"$PERL_INIT_ANSI"', '"$PERL_EXIT_ANSI"', $devices;
! printf "\n";
}
' $opt_f
rm -f $opt_f
***************
*** 659,665 ****
SortBy="&SortBy=CpuTime" # CpuTime is rather unique for each WU in order to prevent fetching duplicate WU-entries from the database (resulting from overlap in two or more fetches) using other key selections such as SentTime.
# (https://www.worldcommunitygrid.org/help/viewTopic.do?shortName=api says "Default is SentTime.")
wcgget
! let avail=`perl -wne 'print if s/\s+"ResultsAvailable": "([^"]+)",\s+/$1/' $TMPFILE.$skipm` # "ResultsAvailable": "395",
let toget=$avail-$opt_l
[ $opt_q -gt 0 ] && rm -f $TMPFILE.$skipm
while [ $toget -gt 0 ]; do
--- 782,789 ----
SortBy="&SortBy=CpuTime" # CpuTime is rather unique for each WU in order to prevent fetching duplicate WU-entries from the database (resulting from overlap in two or more fetches) using other key selections such as SentTime.
# (https://www.worldcommunitygrid.org/help/viewTopic.do?shortName=api says "Default is SentTime.")
wcgget
! avail=`perl -wne 'print if s/\s+"ResultsAvailable": "([^"]+)",\s+/$1/' $TMPFILE.$skipm` # "ResultsAvailable": "395",
! [ -z "$avail" ] && avail=0 # BUGFIX 20-04-2017: Site maintenance
let toget=$avail-$opt_l
[ $opt_q -gt 0 ] && rm -f $TMPFILE.$skipm
while [ $toget -gt 0 ]; do
***************
*** 667,673 ****
wcgget
let toget=$toget-$opt_l # decrease the number of available results
[ $opt_q -gt 0 ] && rm -f $TMPFILE.$skipm
! # BUGFIX 01-03-2017: $TMPFILE$skipm gewijzigd naar $TMPFILE.$skipm
done
}

--- 791,797 ----
wcgget
let toget=$toget-$opt_l # decrease the number of available results
[ $opt_q -gt 0 ] && rm -f $TMPFILE.$skipm
! # BUGFIX 01-03-2017: $TMPFILE$skipm changed into $TMPFILE.$skipm
done
}

----------------------------------------
[Edit 2 times, last edit by adriverhoef at Sep 15, 2017 6:47:42 PM]
[Sep 15, 2017 6:39:17 PM]   Link   Report threatening or abusive post: please login first  Go to top 
adriverhoef
Master Cruncher
The Netherlands
Joined: Apr 3, 2009
Post Count: 2346
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Version 1.36.1 of my script 'wcgresults'

With my script 'wcgresults' you will be able to download recent results from World Community Grid through its API. The script needs Bash to run.

Recently I found out that the command 'boinccmd --get_tasks' (from boinc-client-7.9.2-3.fc27) doesn't list suspended unstarted tasks. After filing a bug report I modified 'wcgresults' to support this kind of missing information (e.g. when using the option -z, -Q, -T or -t).
You may find the current version here.

What's new in version 1.36.1 of 'wcgresults' since version 1.34.7?
- Max. number of WUs per core is now 70 (see option -c).
Example:
$ wcgresults -c
Duration of a workunit (hh:mm:ss or mm:ss or number of seconds): 3h13
You could set your taskcache to 9.38 days.

Duration of a workunit (hh:mm:ss or mm:ss or number of seconds): ^C

- Using the option -f (to record results from finished tasks to a logfile) you will now also
be able to specify the option -s (only selecting results with a certain ValidateState value).
Examples:
$ wcgresults -s0 -df /tmp/pending_validation
$ wcgresults -s4 -df /tmp/pending_verification

- Using the option -f (to record results from finished tasks to a logfile) you will now also
be able to specify the option -w (only selecting tasks that are still in progress).
Example:
$ wcgresults -w -df /tmp/in_progress

[Mar 16, 2018 3:19:54 PM]   Link   Report threatening or abusive post: please login first  Go to top 
TonyEllis
Senior Cruncher
Australia
Joined: Jul 9, 2008
Post Count: 286
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Version 1.36.1 of my script 'wcgresults'

Thanks adriverhoef for the update...

Finally updated from 1.30.1 to 1.36.1 and now making use of the "@" number at the end of the " -Adq" option. Only took me 10 months to get around to upgrade/change my script to make use of it sad
----------------------------------------
----------------------------------------
[Edit 1 times, last edit by TonyEllis at Mar 17, 2018 8:36:47 AM]
[Mar 17, 2018 5:38:45 AM]   Link   Report threatening or abusive post: please login first  Go to top 
adriverhoef
Master Cruncher
The Netherlands
Joined: Apr 3, 2009
Post Count: 2346
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Version 1.36.1 of my script 'wcgresults'

It's very nice to hear from you again, Tony, and that you're still making use of the script. I'm using it every day to check on any abnormalities in the workflow. smile
----------------------------------------
[Edit 1 times, last edit by adriverhoef at Mar 17, 2018 11:35:28 AM]
[Mar 17, 2018 11:33:32 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 65   Pages: 7   [ Previous Page | 1 2 3 4 5 6 7 ]
[ Jump to Last Post ]
Post new Thread