| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Support
Forum: Suggestions / Feedback
Thread: Work unit size and time-outs |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 3
|
|
| Author |
|
|
Chris.B
Cruncher Joined: Aug 17, 2005 Post Count: 1 Status: Offline |
Hello
----------------------------------------Because I am french, I will first write in french and then traduce in english. French version: Ce problème des "work units" trop grosses tombant en time-out pourrait être résolu en mettant en place un système de deux "pipelines". Ces pipelines seraient dimensionnés selon la puissance de l'ordinateur, et ils contiendraient pour l'un les données à traiter, et pour l'autre les données traitées. Lors d'une connexion au serveur via internet, le pipeline de sortie se viderait pendant que celui d'entrée se remplirait à nouveau. Un redimensionnement des pipelines serait envisageable en fonction de l'efficacité de l'unité de traitement. Dans mon idée, les unités auraient intérêt à être les plus petites possibles, leur nombre pouvant varier à souhait à l'intérieur du pipeline.  La connexion à internet devrait pouvoir être initiée par l'utilisateur ou l'agent au moins (par exemple) une fois par jour, dans le but de pouvoir sauvegarder les résultats des traitements effectués depuis la dernière connexion. English version: This problem of too big work-units resulting in time-outs may be solved with a two "pipelines" system. Those pipelines would be sized according to the device power, and they would contain the work units to process for the first one, and the results for the second one. At the time of a connection to the server via Internet, the "output" pipeline would be emptied while the "input" pipeline would fill again. A redimensioning of the pipelines would be possible according to the device's efficiency. In this scheme, work units would have interest to be as smallest as possible, their number varying at will inside the pipeline. Connection to Internet should be able to be initiated by the user or the agent at least (for example) once a day, with an aim of being able to safeguard the results of the treatments carried out since last connection. I hope I will be understood. 
"Les petits ruisseaux font les grandes rivières." (proverbe français)
"The small brooks make the large rivers." (English literal translation of the same French proverb) |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
The WCG dosent have alot of manpower. It is all they can do to run what they have and maybe come up with a linux version of the Grid software.
They also do not seem to use the open source community to take some of the load off thier shoulders. So I would guess your suggestion at best will end up on a rather long to do list. Good Idea anyway!!!!! ciao........ |
||
|
|
Viktors
Former World Community Grid Tech Joined: Sep 20, 2004 Post Count: 653 Status: Offline Project Badges: 
|
Thank you for your suggestion. I too have been thinking about various ways to improve grid efficiency. Ideally, we would like to send shorter work units to slower machines and the longer running ones to faster machines. This would let us balance the work better and reduce storage needs at the server. However, we don't want to make all of the work units very short because this increases our network load. This is because the input data does not decrease in size proportionally to the duration of the work units. Furthermore, machines are not always connected to the internet and very short work units would leave devices idle more of the time. Unfortunately, the computation time required by Rosetta for a given gene is only partially predictable. What you are suggesting is something similar to being able to dynamically adjust the durations of specific work units.
The vendor software we are using does not currently allow us to control everything as we would like, but hopefully we will be able to make improvements in this regard in the future. We have already done some planning for this. |
||
|
|
|