Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 38
Posts: 38   Pages: 4   [ Previous Page | 1 2 3 4 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 4680 times and has 37 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Contribution Target hunting tools - Share!

v1.6a , left side a big "OKAY" as overkill to the big green tick-marked arrow. For the absentminded moments, if a tab is renamed, the program will reverse it [and some will be driven nuts by this ;p]. Though most of the background code does not care, the first 4 characters are critical to the filters working... the sheets-name is picked up by a 'UDF' (User defined Function) to lock the application short-name based filtering.
[Aug 3, 2015 2:07:04 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sgt.Joe
Ace Cruncher
USA
Joined: Jul 4, 2006
Post Count: 7662
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Contribution Target hunting tools - Share!

BTW, whilst applying a much more elegant [code efficient) transform method from seconds to WCG stats time notation, discovered that Office thru 2013 can't handle seconds notations great than 255611462399 [8105:140:23:59:59]. Add 1 second more, and the formula fails with #Value. Most WCG projects exceed this number on the cumulative.


If you break up the formula into individual cells and perform the "mod" and "int" operations separately, you can get around the limitation. The code behind this is(without the "=" sign to designate a formula):
A1:Number in WCG format
B1:LEFT(A1,5)
C1:LEFT(RIGHT(A1,12),3)
D1:LEFT(RIGHT(A1,5),2)
E1:LEFT(RIGHT(A1,5),2)
F1:RIGHT(A1,2)
G1:B1*365*86400+C1*86400+D1*3600+E1*60+F1 (Total time in seconds)
H1:MOD(G1,31536000)
I1:MOD(H1,86400)
J1:MOD(I1,3600)
K1:MOD(J1,60)
G2:INT(G1/31536000)
G3:INT(H1/86400)
G4:INT(I1/3600)
G5:INT(J1/60)
G6:G2&TEXT(G2,":")&G3&TEXT(G3,":")&G4&TEXT(G4,":")&G5&TEXT(G5,":")&K1

This may look a bit cumbersome, but it does, in the end, allow for the conversion of seconds greater than 255611462399 to the WCG format.
Cheers
----------------------------------------
Sgt. Joe
*Minnesota Crunchers*
[Aug 4, 2015 11:53:53 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Contribution Target hunting tools - Share!

Thx, I've stuck to the long code posted in that message i.e.

=INT(H7/SecYr)&":"&RIGHT("00"&INT(MOD(H7/SecYr;1)*365);3)&":"&RIGHT("0"&INT(MOD(H7/SecHr;24));2)&":"&RIGHT("0"&INT(MOD(H7/60;60));2)&":"&RIGHT("0"&MOD(H7;60);2)

So happens OET1 went a few days ago over this critical point of 8105:140 years, thusly plugged in the old [The error check routine kicked it out :]. The hunting sheet will never ever get there, so used the elegant one with the timemask richly.
[Aug 4, 2015 12:51:38 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Still Alive
Advanced Cruncher
Germany
Joined: May 27, 2013
Post Count: 69
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Contribution Target hunting tools - Share!

Nice to see that it is going ahead with the tool.
Although I , I can make only one picture by I look at the pictures of you that you show.
----------------------------------------

AMD 3900X - 21 Threads @ 4,00 GHz - 24/7
Samsung Xcover4 - 4 Threads @ 1.43 Ghz - 24/7
[Aug 4, 2015 5:49:24 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Contribution Target hunting tools - Share!

Thought to get really turbo, navigating the absolute most time consuming part in Office [slowing down VBA execution tremendously], and was doing 'in memory array copying / pasting' before writing which required the current day record notes to be erased after starting a new day. Came out, that erasing only happened time and again on prj tab 1, not on each of the project sheets, so had to get wicked (hat-tip to Useful Gyaan's 2013 post on tricks of the week) which was modded and placed at end of the For Each / Next loop:

With Application.WorksheetFunction.Index(Range(prj & "hist"), 1, Column [NotesOne]).ClearContents

Now, as bonus, does not matter where the project history is, the right record & field is being cleared on each of the processed active project sheets. LoL, being now on record 90, after 45 days of mincing the 12 hourly stats and the limit is 1,048,576 records minus 1 for the headers. The number of archived results suffers same limit, but after 10 years having done 171K+ think this will last a few years biggrin

Moving on... 99.97% complete. (Script: "If the All summary record tallies, everything else need not be marked OK too" to add.)

(This is getting towards redistributable for small crunchers... up to 5 active devices at this time, but easily extendable if the need would arise, really easy... code will be password protected though. ;o).
[Aug 5, 2015 10:48:53 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Contribution Target hunting tools - Share!

Today repurposed the raw data summary area to become the always planned runtime/results per project for validated and pending. Glad the groundwork was done while developing as it proofed a peanut conversion... See red framed area in http://i137.photobucket.com/albums/q210/Seker...utionApp1.6d.png~original

No more tab visiting to see which projects contributions made up the sum. Of course there's cross checks, so totals for the sciences must equal the totals for the active devices, else (No idea why the big grin)

99.99%

ToDo:

1) Make the period duration a input variable, in case 12 hours becomes a chore and 24 hourly run is reliable [There's always that risk of canonical results/too many error results before that time being pulled in a shorter period. So far 8 hours after midnight stats things still are complete].
2) Any other tweaks not thought off till now. cool
[Aug 7, 2015 12:34:49 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sgt.Joe
Ace Cruncher
USA
Joined: Jul 4, 2006
Post Count: 7662
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Contribution Target hunting tools - Share!

I am a bit curious how you eliminate the duplicates on the work fetch. I have a method of dealing with them after the download, but wonder if there is a more efficient way of dealing with them.

Cheers
----------------------------------------
Sgt. Joe
*Minnesota Crunchers*
[Aug 7, 2015 2:20:03 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Contribution Target hunting tools - Share!

Read of others using the API fetch to either have missing items or duplicates, but as yet not encountered this phenomena. For sure I've separated the fetches in InProgress [Cache section], Pending and those with a grant [Outcome 1+3], this essentially cut a single fetch to under 250 as of neither 3 I have more than 250 on the RS pages. Occasionally there's more than 250, for which there's the "Refresh All Result Xml's" button. They are set to fetch 1-250, 251-500, 501-750 and so on. It's usually done in 15-20 seconds. The formula to prevent duplicates, and limit what's included to a 12 hour cycle is:

IF(C3="";"";MOD(SUMIFS(ResultXml1[CpuTime];ResultXml1[DeviceId];C3;ResultXml1[Stats_Period];ActPer);24))

Which computes whole days. The criteria used check out the Device ID and the Stats_Period [using external from-to limits as defined in ActPer [Active Period]. The formulas auto-populate the computed columns... if tables expand, the formula do so too, if table shrinks, the formula ranges do so too. An Excel feature you can tick in table design.

Very easily extensible. This one can handle up to 750 results with a ServerState 5+Outcome X. A simple copy and append for additional fetch blocks just requires an extra data map and this bit

SUMIFS(ResultXml3[CpuTime];ResultXml3[DeviceId];C3;ResultXml3[Stats_Period];ActPer);24))

getting ResultsXml4 instead of ResultXml3 etc.

For the runtime using SumIFS and for results CountIFS which allow very complex criteria definitions. Points/Credits is not set up to be sliced and diced, though the records do include calculating credit/hour and efficiency.

Duplicates is hardly possible in the ways developed as the filter is set to only pick up those that had a modtime change and are in the range of last period start modtime stamp + 43200 seconds. It's all scripted to set the criteria i.e. when hitting Archive, it puts away the Valid/TME's for the last period and starts from the next [have yet another one TME].

Probably confusing, but that's sort of the outline. Not Sql, but the way it works it's been highly reliable and fit for purpose.

v1.6e is just archived... to keep it in design consistency, the pop-up comments now have rounded corners too. smile

One of the ToDo's is automating the additional fetch block creation. It half works, not found the VBA codex yet that allows the injection of the fetch range into the data connectoids... getting some object-mismatch error. No drive to finish it, a nice to have but no must.

In future might split the fetches to each of the possible 9 states, i.e. let WCG server do the filtering for me, chances then being that each of the draws is always under 250.


Oh, make sure to set the API requests to sort on Received time [modtime was requested and would improve, speak reduce, the fetched record needs substantially]. This ensures the newest changed records are at top. If you have more than 250 valids in a 12 hour period [500 a day], that's lots. For me with 3 defined blocks that effectively is covering 1500 results with grant a day. Out of my league, but that is what the app handles till now.

Oh Oh, yes there's the Array formula which you can set by Ctrl+Shift+Enter in Excel which puts a formula into accolades [the array signifier]. A sample looks like this

{=IFERROR(INDEX(ResultXml1[DeviceId];MATCH(0;COUNTIF('ALL Qry'!$C$2:C2;ResultXml1[DeviceId]);0));"")}

Think I posted the sample before. This is Thé base filter to ensure a unique device listing of the queried data i.e. it goes through the whole database and composes a unique [active] device ID list. This can be applied to results too if your method suffers from duplicates.
[Aug 7, 2015 4:33:54 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Contribution Target hunting tools - Share!

Must have caused puzzle ;P

A good example of potential duplicate is this one:

OET1_ 0001063_ xSDGP-L_ rig_ 91016_ 2-- - No Reply 8/5/15 21:54:52 8/9/15 09:54:52 0.00 0.0 / 0.0 < 3) No Reply verifier not replying.
OET1_ 0001063_ xSDGP-L_ rig_ 91016_ 1-- 719 Valid 8/2/15 09:53:09 8/6/15 07:11:31 14.98 51.8 / 51.3 < 2) Verifier, Late Returner
OET1_ 0001063_ xSDGP-L_ rig_ 91016_ 0-- 719 Valid 8/1/15 09:26:50 8/2/15 09:51:56 6.01 50.7 / 51.3 < 1) in PVer

The credit was granted on the 6th when the late returner came in, but the record remained on the RS pages because there was still a task out in the wild because of _1 tardiness. The moment the 3rd copy expired, the modtime changed, so it gets listed on the current day results. Since the record was already on archive , the validator column determines to maintain 'open' state. Come archive time, 'Open' are filtered out. This will be repeated until WCG takes the record and migrates the canonical result into the master database. Since on coming cycle the modtime will fall outside the 'current' cycle, the next part of the filter, the record will fall out of active record view anyhow i.e. no distraction.

Sample formula:

=IF(OR(AND([@ValidateState]=3;[@ModTime]>PeriodStart;[@ModTime]<=PeriodEnd);AND([@Status]="Invalid";[@ModTime]>PeriodStart;[@ModTime]<=PeriodEnd;ISNA(MATCH([@Name];Archive[Result Name];0))) ;AND([@Granted]>0;[@ModTime]>PeriodStart;[@ModTime]<=PeriodEnd ;ISNA(MATCH([@Name];Archive[Result Name];0))));ActPer;"Open")

A next version will just do an index/match check based on a matrix build off server state / validate state and outcome. Just finished cycle 97, a push of 1 button (I initiates II for me), and all turned green again, so pushed III, archive, and ready we are for tomorrow morning to hit the buttons again, in that order. biggrin

As for beyond 100% complete, today added a piece of script to overlay the -highly distracting- red comment indicators with small gray triangles [hat-tip to Dana Louis of Contextures who coded this up in 2009]. Tweaked it a little to get -perfect- overlays, to the tenth of a pixel [0.1], and slide it 0.3 pixel left to not sit on top of any cell borders. This is how precise the rendering can be controlled. The color index allowed picking off 56 colors and took gray over blue. Modded to VBA further to remove any old triangles before putting new in place, [was ending up with a stag of them of varying colors, sizes, positions, while trialling for optimization].

Image of v1.6e , this time in full-screen view, so hit zoom on a wide display to feast the eye whistling

The ranking rot has stopped, courtesy of the multi-boot which is crawling to get that 2.656GB W10 Pro downloaded via a bridged connection and the Results/Cycle curve jumped [right top graph], as the machine is doing FAAH shorties while the upgrading takes place... and not exactly in an hour, it's been running since several days before MS set the flag to start the process. biggrin
[Aug 9, 2015 1:58:33 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Contribution Target hunting tools - Share!

Nothing much happening, other than realizing this app is dependent on Office 64 on OS in 64 bit such as using the sound drivers, which limits redistributability [curious how MS themselves recommends to install the 32 bit version 'for compatibility'].

Tweaked the Recon Nook to label the nightly spill if it was the nightly spill making up any difference. Under the hood, if a project has not had a transaction for 24 hours, no new record is being created in the history, just as UD's stats engine does for the projects. Keeps things a little tighter, most prominently the Beta history... every three+ months a few lines.
[Aug 17, 2015 5:30:35 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 38   Pages: 4   [ Previous Page | 1 2 3 4 | Next Page ]
[ Jump to Last Post ]
Post new Thread