As we've experienced in the last few months, somehow 3rd party statistics websites appear to have trouble to get the -monster- host.gz file [currently 390MB compressed, > 2GB uncompressed]. Myself have experienced it occasionally needing 2-3 attempts before succeeding a download. As it is, the content is relating to 90% idle devices, not having had results in past 30 days and 10% active devices [round about 250,000]. The proposal is to generate an additional file only including those devices that have had a result validated in the past 24 hours [UTC of course]. The file could be called 'hostactive.gz'. What this would help doing is to simply add/draw difference against the content of the locally stored/processed stats. Instead of downloading 390MB, way less than 39MB would be enough [daily active is less than last 30 day active]

The second file would facilitate to also compute prior day position as when the host.gz download would fail on the current day, the current day hostactive.gz less the same devices in the host.gz of previous day [for active devices], would allow to still determine the device stats for the last day. It could even facilitate a daily contribution reconstruction over a longer period if retaining copies of the 'small' hostactive.gz

-Certainly- this would end silly numbers like now showing on BOINCStats, since whenever the download truncation started in earnest a few month ago...

Users 485,960
Active users 66,119 (13.61%)
Hosts 2,424,843
Active hosts 1,816,957 (74.93%)
Teams 23,891
Active teams 7,177 (30.04%)

Willy, de Bok and others can take benefit from this, if they want to. If 10 stats keepers do not need to download the 390MB file daily, it means to WCG reducing bandwidth load by 3.5GB, and rising.

Just another idea [to save money too ;)]