Right - that's it dc! I'm firing up my two unused laptops - only dual core but should add another 4 days / day to my run time - the war is on

Yes, you're absolutely right RTS48. I can put an explanation at the top of the list, or maybe have another 3 lists of team members who are in the top 5000, or something? I hope to get the first draft of my head-2-head comparator up and running this weekend, so that you'll be able to see how far behind me you are in the current runtime figures

EDIT 1: I've now added 3 more tables for your delectation and delight.
EDIT 2: I've added the most basic compare feature that I could in order to prove the concept. Much more to come, including remembering what you've picked after you refresh the page.

Congrats Thargor. 125 years or 150 million points next? My next one is a mere 25 years, and I'm going to have to wait 3 weeks or so for that.

Good morning everyone.

Congrats Thargor.

dcrobinson stats are looking good.

Thanks for the compliments Rod (aka RTS48) and domonijo.

Here are the full details of how it works, including some links in case you fancy some light reading .

I have my own server (sat under my desk at home) on which I run several virtual machines . I have added one especially for the stats, running the latest Debian "Jessie" GNU/Linux operating system.

The WCG has a number of webpages containing the statistics by team and member. On many of these, if you add "&xml=true" to the end of the URL you get the same data in structured XML format . For example, https://secure.worldcommunitygrid.org/team/viewTeamInfo.do?teamId=L721SPD4BN1 is the main team UK page, but https://secure.worldcommunitygrid.org/team/vi...=L721SPD4BN1&xml=true gives you the same page but just the raw data in a structured format. Notice also that at the bottom of the XML page there are links to yet more data, enabling you to drill down through the structure.

I have written some software (it's what I do for a living), in the Java language, which sends off web requests to the WCG site and pulls down the data in XML format. As an aside, Java gets a bad press for security issues. However, most of these are related to the web browser plugin... the desktop and server versions of Java are pretty good and are widely used (for example, most Android applications, and many big websites are powered by Java).

I process the XML data using something called JAXB which makes it really easy to turn a wodge (technical term) of XML data into a bunch (another technical term) of nicely linked-together hierarchical Java objects. I then traverse this data and shove various bits of it into an SQL database. I do a small amount of upfront calculation at this point (very little). This takes about 10 minutes, but I only pull down the team data for the top 250 teams, and the member data for just team UK. Originally I was ingesting data for 17,000+ teams, but that was largely pointless and took over 90 minutes.

The slurping up of the data runs at 02:30 (UK local time) every night. I have deemed this to be a sufficient time after the midnight UTC stats update. Once the data ingest has completed, it notifies the second part of the solution that something has changed.

A second Java program serves out the web pages, using an embedded web server called Jetty . This uses a technology called Java servlets . On startup, this program does a whole lot of calculations (such as working out the rankings and movements), and stores the data in memory. This involves a lot of SQL queries and some basic arithmetic. Being a computer, these calculations take about a second. It repeats these calculations nightly when notified that new data has arrived. Most of the data displayed on the website is just a manifestation of this precalculated data. However, some of the member comparison data is directly obtained from the SQL database on request.

What you see in your web browser is an AJAX website built with the help of jQuery Mobile . This is a JavaScript code library for building web interfaces for multiple screen resolutions, so the stats site is supposed to work well on mobile devices (note that JavaScript has little in common with Java other than the name). This is the one side of the technology that's new to me, so it's not perfect, yet. The graphs are rendered using something called Flot , which is an extension to jQuery.

I hope some of that is interesting. FYI, I've written FORTRAN 66 myself in the past (good old Holerith strings), but when I started at university in 1983 the card reader was just about to be pensioned off.