This is still happening, why!

A lot of wasted time.

Result Log

Result Name: ZIKA_ 000005583_ x1nb7_ HCVJ4_ RNAPol_ wRNAand2Mn_ chnA_ 0233_ 0--

Project Name: OpenZika
Created: 05/29/2016 13:03:17
Name: ZIKA_000005583_x1nb7_HCVJ4_RNAPol_wRNAand2Mn_chnA_0233
Minimum Quorum: 1
Replication: 1

Linux 3.16.0-71-generic 705 Error 5/29/16 21:35:40 5/31/16 10:48:59 11.59 40.6 / 0.0

{ cut top bit off }

[20:27:57] ./ZINC05649550.pdbqt size = 23 5 ../../projects/www.worldcommunitygrid.org/7d9bc432a983d11998f072164e7d0f83.pdbqt size = 5514 0
[20:29:45] Finished task #349 cpu time used 109.326396
20:29:45 (4542): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
<file_name>ZIKA_000005583_x1nb7_HCVJ4_RNAPol_wRNAand2Mn_chnA_0233_0_r966012209_0</file_name>
<error_code>-131 (file size too big)</error_code>
</file_xfer_error>

</message>

@Uplinger, help! We're still having errors on some even bigger returns. The 8MB limit is not sufficient for them. Can you please increase the max size again?

Here's some examples:
31-May-2016 05:44:54 [World Community Grid] Computation for task ZIKA_000001130_x1nb7_HCVJ4_RNAPol_wRNAand2Mn_chnA_0331_0 finished
31-May-2016 05:44:54 [World Community Grid] Output file ZIKA_000001130_x1nb7_HCVJ4_RNAPol_wRNAand2Mn_chnA_0331_0_r575309385_0 for task ZIKA_000001130_x1nb7_HCVJ4_RNAPol_wRNAand2Mn_chnA_0331_0 exceeds size limit.
31-May-2016 05:44:54 [World Community Grid] File size: 8594868.000000 bytes. Limit: 8388608.000000 bytes

31-May-2016 10:22:30 [World Community Grid] Computation for task ZIKA_000001059_x1nb7_HCVJ4_RNAPol_wRNAand2Mn_chnA_0323_4 finished
31-May-2016 10:22:30 [World Community Grid] Output file ZIKA_000001059_x1nb7_HCVJ4_RNAPol_wRNAand2Mn_chnA_0323_4_r225492679_0 for task ZIKA_000001059_x1nb7_HCVJ4_RNAPol_wRNAand2Mn_chnA_0323_4 exceeds size limit.
31-May-2016 10:22:30 [World Community Grid] File size: 9002594.000000 bytes. Limit: 8388608.000000 bytes

30-May-2016 19:13:04 [World Community Grid] Computation for task ZIKA_000000824_x1nb7_HCVJ4_RNAPol_JustProt_chnB_0323_0 finished
30-May-2016 19:13:04 [World Community Grid] Output file ZIKA_000000824_x1nb7_HCVJ4_RNAPol_JustProt_chnB_0323_0_r2031644808_0 for task ZIKA_000000824_x1nb7_HCVJ4_RNAPol_JustProt_chnB_0323_0 exceeds size limit.
30-May-2016 19:13:04 [World Community Grid] File size: 9002360.000000 bytes. Limit: 8388608.000000 bytes

I have grabbed the work units that members have returned an issue due to file upload size being too large. I am running them on our alpha grid to see what size they are actually becoming. Out of the 700k results I have in returned status on the database, only 132 workunits were affected. I am hopeful to get a better idea of the size needed and will adjust accordingly.

Thanks,
-Uplinger

@Uplinger - here's the size for some of the errors we've seen in the last few days (size in MB/WU):

8.3145 ZIKA_000000985_x1nb7_HCVJ4_RNAPol_wRNAand2Mn_chnA_0342_2
8.4477 ZIKA_000005333_x1nb7_HCVJ4_RNAPol_JustProt_chnA_0175_1
8.1967 ZIKA_000001130_x1nb7_HCVJ4_RNAPol_wRNAand2Mn_chnA_0331_0
8.5855 ZIKA_000001059_x1nb7_HCVJ4_RNAPol_wRNAand2Mn_chnA_0323_4
8.5853 ZIKA_000000824_x1nb7_HCVJ4_RNAPol_JustProt_chnB_0323_0
8.7258 ZIKA_000000977_x1nb7_HCVJ4_RNAPol_wRNAand2Mn_chnA_0325_1
8.4474 ZIKA_000005256_xNS5_ModelSE_min_0115_1
8.3795 ZIKA_000005208_xNS5_ModelSE_min_0237_0
8.5468 ZIKA_000005175_xNS5_ModelSE_min_0372_0
8.6444 ZIKA_000000781_x1nb7_HCVJ4_RNAPol_JustProt_chnB_0336_2
8.6444 ZIKA_000001016_x1nb7_HCVJ4_RNAPol_wRNAand2Mn_chnA_0336_0
8.3141 ZIKA_000000750_x1nb7_HCVJ4_RNAPol_JustProt_chnB_0342_2
8.3141 ZIKA_000000750_x1nb7_HCVJ4_RNAPol_JustProt_chnB_0342_0

Size is size in bytes in the log / 1024^2.

Biggest one in this list is 8.7258 MB. I'd imagine 10 or 12 MB would suffice. Is there a benefit to making this restrictive?

Ok, So a few things we have checked.

1. There is a hard limit of 350 jobs per work unit. A majority of the work units run less than that, most below 100. For some reason there is a group of ligands that are estimated incorrectly and have not adjusted in the database that we use to estimate.

2. The file upload limit is being set to the uncompressed file that is sent back. The upload compression is about 1:8 ratio.

With these two things in mind and the fact it was only like 130 work units, I have increased the upload size to 16MB for the time being. This should help us get past these file size issues and get some more datapoints to estimate these batches better. Currently work units are running an estimated 1h 51m, which is pretty good since we are targeting 2 hour run times at the moment.

Thanks,
-Uplinger