Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go ยป
No member browsing this thread
Thread Status: Active
Total posts in this thread: 61
Posts: 61   Pages: 7   [ Previous Page | 1 2 3 4 5 6 7 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 22420 times and has 60 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: problem with daily update?

ok - team totals for the 12th are good - but team totals for 10/07 are still 1/2 what they should be!
[Oct 13, 2017 12:50:34 AM]   Link   Report threatening or abusive post: please login first  Go to top 
keithhenry
Ace Cruncher
Senile old farts of the world ....uh.....uh..... nevermind
Joined: Nov 18, 2004
Post Count: 18665
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: problem with daily update?

Well, the normal EOD update for today should be responsible for the team totals being right for today. Until Jonathan finishes running the needed imports (he still has yet to say they are done after all), the 7th's team numbers probably won't be. Clearly, the move to the cloud modified the stats process behind the scenes. Past stats snafus never required this much work. Jonathan has mentioned determining which imports need to be run. Speculation here but I would imagine that running the wrong imports would quickly make things much worse. It could well be that determining which specific imports were needed involved a lot more work than simply running them. Since the user ones appear to be done, progress is happening. The team ones are probably still running. There may even be more after that than will need to be run. When it's all done and over with, I imagine Jonathan will give us a bit of a rundown before he passes out. biggrin
----------------------------------------
Join/Website/IMODB



[Oct 13, 2017 1:14:10 AM]   Link   Report threatening or abusive post: please login first  Go to top 
TPCBF
Master Cruncher
USA
Joined: Jan 2, 2011
Post Count: 1960
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: problem with daily update?

shock
I'm surprised to see how many importance some people give to the stats. Ok, we can admit that they are important for the challenge of the team, or for its own enjoyment, but....
What's the more important ? have good stats or help the research ?
At least from my POV, it is not as much about the stats by them self, but the possibility that those WUs that make up the stats (or the lack there of) might indicate that there is no help for the research as they could have ended up in the eternal bit bucket...
For me, the most important aspect of the stats is that there is smooth "help for the research". Any unusual deviation of those stats means either that there is an issue on the side of my hosts, which would be something I have to track down and take care of, or the possibility that there is a snafu on the side of the project, with the possibility that those WUs are lost and therefor of no help for the research...

Ralf
----------------------------------------

[Oct 13, 2017 1:16:27 AM]   Link   Report threatening or abusive post: please login first  Go to top 
uplinger
Former World Community Grid Tech
Joined: May 23, 2005
Post Count: 3952
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: problem with daily update?

ok - team totals for the 12th are good - but team totals for 10/07 are still 1/2 what they should be!


We will need to look into why that grouping didn't get updated at the same time.

I did a check against your stats from the 6-8 and it looked normal for individual stats:

2017-10-08 00:00:00 144 672617 36715
2017-10-07 00:00:00 120 591087 32609
2017-10-06 00:00:00 111 653999 28099

Thanks,
-Uplinger
[Oct 13, 2017 2:44:31 AM]   Link   Report threatening or abusive post: please login first  Go to top 
uplinger
Former World Community Grid Tech
Joined: May 23, 2005
Post Count: 3952
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: problem with daily update?

Yes, I'm seeing numbers for the 7th for me that are basically double what it has been showing. With half a day's data missing, that fits. So, the 7th looks okay. I'm a tad concerned that today's numbers are also abnormally high - by about 50 percent but not the same 50 percent as on the 7th. I haven't had a jailbreak - PVals are the same numbers as yesterday so any that got out have been replaced by new WUs from today. PVer is actually up from yesterday. I can't help but wonder if there is a connection with the import process (which may not be done). If I still see this after the EOD update tonight, then I'll be concerned. Right now, I just really can't explain it.

As for the problem from the 7th, the real issue is what is going to be done to RCA the import failure, add proper monitoring so that the admins get notified of any future failures and are they going to fix the process so that the date/timestamps on the stats do NOT update if the import fails.


To get the stats to import properly from the 7th, I had to set the scripts that run this back to the 6th and import the stats until that moment. This means that there was a mini end of day stats run done to import everything from the 7th. One thing I overlooked when fixing the individual tasks was that the timestamp that needs to be set for team imports is differnt than the one for user imports. This is why the team stats weren't updated properly. I am working to import this data into their correct tables in the database now.

Just so everyone knows. We have all the individual task stats for BOINC projects on a daily basis from day one with BOINC. The data was not lost or misplaced, just the scripts that handle what was new in that table on the 7th failed, but still marked the stats run complete. Thus when the next time stats ran, it only picked up the data from the 8th onward.

Thanks,
-Uplinger
[Oct 13, 2017 3:11:17 AM]   Link   Report threatening or abusive post: please login first  Go to top 
uplinger
Former World Community Grid Tech
Joined: May 23, 2005
Post Count: 3952
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: problem with daily update?

Merko,

Just to let you know it looks like that team stats are populating properly for your team.

2017-10-08 00:00:00 4 4 211 1431044 55970
2017-10-07 00:00:00 3 3 185 1313703 48967
2017-10-06 00:00:00 3 3 172 1337936 44243

The 7th now shows 185 results instead of the 93 it was before.

Thanks,
-Uplinger
[Oct 13, 2017 3:12:45 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: problem with daily update?

ok- so when will they show up in MY team totals?
[Oct 13, 2017 4:35:53 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: problem with daily update?

Uplinger, many thanks for the work you have done, my stats for the 07/10/17 (Thursday) now seem what I expected.

It was possibly one of those s**t happens kind of thing. Whatever it was many thanks for your and your teams work in correcting it. smile smile smile
[Oct 13, 2017 7:52:54 AM]   Link   Report threatening or abusive post: please login first  Go to top 
RTS48
Veteran Cruncher
Bolivia
Joined: Aug 2, 2009
Post Count: 1350
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: problem with daily update?

Thanks Uplinger - I had a look today and my Saturday Stats had been repaired. Now I only need to find out why my points haul has plummeted!
----------------------------------------
Rod Peel
Santa Cruz
Bolivia
South America

,
,
[Oct 13, 2017 1:18:39 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: problem with daily update?

Thank You Very Much Uplinger - My teams totals are now correct - what happened, was it an error in the coding or a hiccup?
[Oct 13, 2017 1:55:23 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 61   Pages: 7   [ Previous Page | 1 2 3 4 5 6 7 | Next Page ]
[ Jump to Last Post ]
Post new Thread