Maybe there's cross-result learning being applied before generation of the next seed? You're not the first to observe AsyncRE to be stuck in PVal, albeit there was no come back if that earlier report got resolved: https://www.worldcommunitygrid.org/forums/wcg/viewpostinthread?post=554089

Just checked Results Status and I currently have 28 FAH2 wu in PVal covering the past eight hours. The oldest was returned at 11/13/17 06:09:13. It is now coming up on 14:00:00. Further checking shows that these are scattered among the FAH2 wu I have returned today. They are not part of a single block. It would seem that the validators do not work on a FIFO basis as I would expect with single replication wu. My thinking at this point is that the FAH2 validators are simply unable to handle the current volumes everyone is returning.

Greetings,

Short answer to subject: Yes.

Long Answer: The validators have some extra checks in it to make sure things are progressing well. When something is off or needs to be investigated, it stops running. This is to prevent things from getting out of hand and to see why in a specific case where it is assumed all would be fine, isn't. In this particular case, when a result is returned, we check the original input file for some data to help us determine what is needed to be validated and what is expected. It has failed to read this input file. Thus, STOP!!! Alerts are sent. It happened 3 times over the weekend and I'm working on fixing that permanently. Since it was the weekend and a vacation, I opted on fixing up the input file for those results manually since that took only a few minutes. I will be working towards the permanent fix today.

Thanks,
-Uplinger

Thank-you for the details Keith -- much appreciated!

Greetings,

Just an update. I am working on deploying the changes now. Currently all FAHB validators are disabled until the fix is completed.

Thanks,
-Uplinger