Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 18
Posts: 18   Pages: 2   [ Previous Page | 1 2 ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 3605 times and has 17 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Stats seem to be running longer than usual

The Unavailable screen was showing 47 minutes when I looked at it during the 0000UTC stats update. However, it became available prior to the 47 minutes. The time varies considerably and the message is only somewhat accurate.
[Sep 20, 2016 3:10:04 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Stats seem to be running longer than usual

Kevin, the timestamp printed on MC does not match the actual cutoff time. The stamp today says 12:06:02 and had an infamous 'spill' difference [short] of 0:20:47:28 which turns out to be this result with a lastmod timestamp of 12:06:11, 9 seconds later.

HST1_007799_000083_MT0005_T300_F00055_S00011_1 Valid 543.9975944 20.79107285 0:20:47:28 20.90606117 0 346.1631912 16.64960696 99.45% 3525069 1474632371 1 16-09-15 08:59:49 16-09-24 12:03:12 16-09-14 12:03:12 5 1 16-09-23 12:06:11

Why is this problematic to me? Because when the Result Status page pull again is done at 00:06 and some seconds after, this result could already be gone i.e. necessitated to manually change the stamp, so my tool can include it for the morning period and archive it.

Why this is problematic to you? You wrote that the next BOINC stats export picks up from last timestamp... but then you're going to tell us, that what is printed on the can, is not really used by your system, which brings be to the problem of having to guess what is and what is not, where one part such as the statistics history truly uses the 00:00:01 to 24:00:00 UTC day to build the daily MC stats, but on the other hand the MC main page includes that 6 minute spill, with variable seconds after the 6th minute, which leads me to the request: Could we get the real cutoff time printed on the stats pages? In that case, no second guessing and my tool could simply read the seconds exact time from the website and use that to calculate what's in and what's out in credited results.

Of course, the alternate is to pause the validator for that 00:00-00:06:15 timeframe, to take it with leeway, and forever we'd be rid of this silly problem... all stats pages adding up and x-tallying. Of course, the 2bd alternate is to set the cut-off altogether at 00:00:00, and then no validator pausing is needed (does anyone still remember why at all that spill period exists?)

thanks.

Doneske, now that I've solved a mechanized sign-in problem, so noticed today the MC history updated at 12:40 UTC, maybe a few minutes earlier, whilst I was hammering the test loop. Used to be around 12:25-12:30 for the noon run. Not seen the flip happened on the main page, but think we're in with permanent slower times, be it, it's curious to happened within days... something changed. confused
[Sep 23, 2016 4:02:37 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Stats seem to be running longer than usual

That's been my point from the beginning (it didn't just grow that way over time but grew suddenly without a corresponding increase in returned results). 12:40 UTC, isn't too bad. Last week it was several (5 -10) minutes past 13:00 UTC on a few days.
[Sep 23, 2016 6:04:26 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Stats seem to be running longer than usual

Sorry for confusion. MC Stats History is the first element to show if stats are progressing, the MC Main I did not see ("Not seen the flip happened on the main page"), which is what you've been watching. That I guess is still around your observed time.

The way I'm interpreting the knreed times list is, that both member MC stats history and and even MC main update before the nightly lockout, certainly the first. Just not going to sit up and watch it at 3 AM CET... ways past my sleep time. Global will be much after the member data is updated (without the timestamp indication)
----------------------------------------
[Edit 1 times, last edit by SekeRob* at Sep 23, 2016 6:27:38 PM]
[Sep 23, 2016 6:25:29 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Stats seem to be running longer than usual

At around 12:27 UTC on The Device Statistics page

Statistics Last Updated: 9/26/16 12:06:02 (UTC) [0 hour(s) ago]

It's of course the last place to look if stats are progressing, but this really caused me almost to tear at the corners of the ear to ear smile, just briefly, as none of the other MC page were clown

(It's only because I'm testing heavily crowbarring, size caliber 50", a web page into an office table)
[Sep 26, 2016 12:39:58 PM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Stats seem to be running longer than usual

There are two bits of data here.

The stats history that drives everything for stats on the website EXCEPT the results status page and the "last result returned" field for a user.

Here is a little more detail about the stats process:
1) We added a device stat history table in the BOINC database. When the validator runs on a result, and the result becomes either valid or invalid, then it will add/update an entry on the device stat history table in the database.
2) The device stat history table in the BOINC database stores the date (not a timestamp), the application id, and the amount of points, cpu time and count of results.
3) When the stats process runs, it looks at the timestamp that the stats was last run and turns that into a date. It then exports all of the stats for that date forward in the process. These are then used to update all the other levels of stats (overwriting a previous record for that day if necessary) on the website database.

The "last result returned" runs every three hours and simply looks at results returned since the last time it ran and exports the last result by userid and updates that field on the user table in the website database.

The results returned page shows the data directly from the BOINC db. As a result, that data is the actual live data.
[Sep 26, 2016 1:19:14 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SekeRob
Master Cruncher
Joined: Jan 7, 2013
Post Count: 2741
Status: Offline
Reply to this Post  Reply with Quote 
Re: Stats seem to be running longer than usual

Hey Kev, the 'last result returned" stamp we knew is frequently, 3 hour interval, but do the runtime cumulatives per-device etc do so too? That would be 'live', but not seen it :O) It would upset my hunting tool , if runtimes etc would refresh too at that pace.

Still it's puzzling how that top of page timestamp updated well before the rest.
[Sep 26, 2016 3:05:57 PM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Stats seem to be running longer than usual

Stats at the user and host level are updated twice a day.

The "stats last updated" actually consists of multiple values:
  • Device
  • User
  • User on a Team
  • Team
  • Country
  • Project
  • Global
The stats process is divided into three chunks. Export from BOINC, loading of the twice a day data points and loading of the once a day day points.

At the end of the export from BOINC step, the value for devices is updated.

At the end the twice a day load step the following are updated:
  • User
  • Project
  • Global

At the end of the once a day load step the following are updated:
  • User on a Team
  • Team
  • Country

----------------------------------------
[Edit 1 times, last edit by knreed at Sep 27, 2016 1:46:14 PM]
[Sep 27, 2016 1:45:39 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 18   Pages: 2   [ Previous Page | 1 2 ]
[ Jump to Last Post ]
Post new Thread