Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 29
Posts: 29   Pages: 3   [ Previous Page | 1 2 3 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 4011 times and has 28 replies Next Thread
[P3D]Magiceye04
Cruncher
Joined: Jul 5, 2008
Post Count: 38
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Unable to download tasks for a week

I do not abort any WU, I just let them try do finish the download.
But the only thing that I see is the list of "Error" WUs is increasing every day by some dozen. Not a single WU starts crunching.
And I also allow all projects, even MCM only have errors.
Have you ever tried to retry the transfers? Sometimes all it takes is one kick, but definitely today (Monday), thinks seem to go somewhat better than in recent days, as someone at WCG towers might have kicked the right tire.
While the current situation is certainly a pain in the posterior, it isn't so as you could can't get ANY WUs at all either...


Ralf

Yes, every 300seconds.
[Nov 12, 2024 6:22:50 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Link64
Senior Cruncher
Joined: Feb 19, 2021
Post Count: 206
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Unable to download tasks for a week

It's not the same, but likely related to that. Unfortunately, like all other of those huge txt files, WCG didn't make this file "sticky", so I guess many people are redownloading it now after either running out of WUs or after they had the great idea to abort all of them.
There is nothing to "make sticky". Each ARP1 WU consists of about a dozen different files, some with just a few KB, a lot more with somewhere between 11 and 15MB and one file about 47-79MB. These are all individual, unique files, with slight variation in size. About half of those files start with ARP_* (the WU name) and the other half are files with random 32 hexadecimal characters as their name...

Ralf

e55b6bdba4ed0b4b6e315c6767d68e3f.txt is a 45.9MB file for MCM, which we are using for every WU since few months and which is deleted every time a computer runs out of MCM work. Before this file was released, we had a ~102MB file with different name and it was also used for every WU. Just the small 915 bytes files are different for every WU.
----------------------------------------

[Nov 13, 2024 5:15:01 PM]   Link   Report threatening or abusive post: please login first  Go to top 
TPCBF
Master Cruncher
USA
Joined: Jan 2, 2011
Post Count: 2173
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
angry Re: Unable to download tasks for a week

It's not the same, but likely related to that. Unfortunately, like all other of those huge txt files, WCG didn't make this file "sticky", so I guess many people are redownloading it now after either running out of WUs or after they had the great idea to abort all of them.
There is nothing to "make sticky". Each ARP1 WU consists of about a dozen different files, some with just a few KB, a lot more with somewhere between 11 and 15MB and one file about 47-79MB. These are all individual, unique files, with slight variation in size. About half of those files start with ARP_* (the WU name) and the other half are files with random 32 hexadecimal characters as their name...

Ralf

e55b6bdba4ed0b4b6e315c6767d68e3f.txt is a 45.9MB file for MCM, which we are using for every WU since few months and which is deleted every time a computer runs out of MCM work. Before this file was released, we had a ~102MB file with different name and it was also used for every WU. Just the small 915 bytes files are different for every WU.
Wrong! Dead wrong. This is NOT an MCM1 file, but one of a set of 11 files that are part of a ARP1 WU download. It is unique (and varying in size, anywhere from 44 to 49MB) to each ARP1 WU...
Each ARP1 WU down load consist of 4 files starting with the ARP1 WU name and and ending in _input, with 1.74KB in size plus 3 ARP1*_input_d01 though_d03 files, all of a size of about 11.xMB, as well as 7 files with 32 random hexadecimal number characters, with 3x in the 650KB range and the rest varying between roughly 12MB and 18MB plus one file in 44-49MB size range.

This is NOT a static MCM1 "master" file that is the same each and every time and for every one...

Ralf
----------------------------------------
[Edit 2 times, last edit by TPCBF at Nov 13, 2024 5:40:44 PM]
[Nov 13, 2024 5:35:12 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Crystal Pellet
Veteran Cruncher
Joined: May 21, 2008
Post Count: 1403
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Unable to download tasks for a week

It's not the same, but likely related to that. Unfortunately, like all other of those huge txt files, WCG didn't make this file "sticky", so I guess many people are redownloading it now after either running out of WUs or after they had the great idea to abort all of them.
There is nothing to "make sticky". Each ARP1 WU consists of about a dozen different files, some with just a few KB, a lot more with somewhere between 11 and 15MB and one file about 47-79MB. These are all individual, unique files, with slight variation in size. About half of those files start with ARP_* (the WU name) and the other half are files with random 32 hexadecimal characters as their name...

Ralf

e55b6bdba4ed0b4b6e315c6767d68e3f.txt is a 45.9MB file for MCM, which we are using for every WU since few months and which is deleted every time a computer runs out of MCM work. Before this file was released, we had a ~102MB file with different name and it was also used for every WU. Just the small 915 bytes files are different for every WU.
Wrong! Dead wrong. This is NOT an MCM1 file, but one of a set of 11 files that are part of a ARP1 WU download. It is unique (and varying in size, anywhere from 44 to 49MB) to each ARP1 WU...
Each ARP1 WU down load consist of 4 files starting with the ARP1 WU name and and ending in _input, with 1.74KB in size plus 3 ARP1*_input_d01 though_d03 files, all of a size of about 11.xMB, as well as 7 files with 32 random hexadecimal number characters, with 3x in the 650KB range and the rest varying between roughly 12MB and 18MB plus one file in 44-49MB size range.

This is NOT a static MCM1 "master" file that is the same each and every time and for every one...

Ralf
Ralf you're wrong!

e55b6bdba4ed0b4b6e315c6767d68e3f.txt is a file belonging to the MCM1 project and is only downloaded when your MCM1 tasks are all returned and you request new work for MCM1.
[Nov 13, 2024 6:36:02 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Link64
Senior Cruncher
Joined: Feb 19, 2021
Post Count: 206
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Unable to download tasks for a week

Than why do I have it on my current computer, which has never seen an ARP WU?

This is the content of "dataset-curatedOvarian_EarlyLate_v1.0.txt", which can be found in every MCM slot dir:

<soft_link>../../projects/www.worldcommunitygrid.org/e55b6bdba4ed0b4b6e315c6767d68e3f.txt</soft_link>


The beginning of the file:

ID curatedOvarian_EarlyLate_v1.0
TargetType binary Ngenes 10229 Nsamples 711
TargetLabels early late
* GSM368671 GSM368674 GSM368675 GSM368678 GSM368679 GSM368683 GSM368689 GSM368697 GSM368700 GSM368703 GSM368706 GSM368707 GSM368709 GSM368710 GSM368711 GSM368724 GSM368725 GSM368728 GSM368729 GSM368730 GSM368731 GSM368732 GSM461348 GSM461349 GSM461350 GSM461351 GSM461352 GSM461353 GSM461354 GSM461355 GSM461356 GSM461357 GSM461358 GSM461359 GSM461360 GSM461361 GSM461362 GSM461363 GSM461364 GSM461365 GSM461366 GSM461367 GSM461370 GSM461371 GSM461373 GSM461374 GSM461375 GSM461376 GSM461377 GSM461378 GSM461381 GSM461382 GSM461383 GSM461384 GSM461385 GSM461386 (...)

This is not ARP1, it's dataset for the current batch of MCM1 and it's the same file since months, my current copy has beed downloaded the 12th June (I back it up outside the WCG dir and hardlink it back every time BOINC deletes the file, than BOINC just checks the file and skips the download).
----------------------------------------

----------------------------------------
[Edit 2 times, last edit by Link64 at Nov 13, 2024 6:48:56 PM]
[Nov 13, 2024 6:42:11 PM]   Link   Report threatening or abusive post: please login first  Go to top 
TPCBF
Master Cruncher
USA
Joined: Jan 2, 2011
Post Count: 2173
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Unable to download tasks for a week

It's not the same, but likely related to that. Unfortunately, like all other of those huge txt files, WCG didn't make this file "sticky", so I guess many people are redownloading it now after either running out of WUs or after they had the great idea to abort all of them.
There is nothing to "make sticky". Each ARP1 WU consists of about a dozen different files, some with just a few KB, a lot more with somewhere between 11 and 15MB and one file about 47-79MB. These are all individual, unique files, with slight variation in size. About half of those files start with ARP_* (the WU name) and the other half are files with random 32 hexadecimal characters as their name...

Ralf

e55b6bdba4ed0b4b6e315c6767d68e3f.txt is a 45.9MB file for MCM, which we are using for every WU since few months and which is deleted every time a computer runs out of MCM work. Before this file was released, we had a ~102MB file with different name and it was also used for every WU. Just the small 915 bytes files are different for every WU.
Wrong! Dead wrong. This is NOT an MCM1 file, but one of a set of 11 files that are part of a ARP1 WU download. It is unique (and varying in size, anywhere from 44 to 49MB) to each ARP1 WU...
Each ARP1 WU down load consist of 4 files starting with the ARP1 WU name and and ending in _input, with 1.74KB in size plus 3 ARP1*_input_d01 though_d03 files, all of a size of about 11.xMB, as well as 7 files with 32 random hexadecimal number characters, with 3x in the 650KB range and the rest varying between roughly 12MB and 18MB plus one file in 44-49MB size range.

This is NOT a static MCM1 "master" file that is the same each and every time and for every one...

Ralf
Ralf you're wrong!

e55b6bdba4ed0b4b6e315c6767d68e3f.txt is a file belonging to the MCM1 project and is only downloaded when your MCM1 tasks are all returned and you request new work for MCM1.
Sorry, while there is A larger "master file" that gets downloaded if you have not crunched on MCM1 for a while, this is NOT THIS FILE.

Please show me a screenshot of two consecutive MCM1 downloads where this file is being download, the same one, twice (or multiple times) in a row!

When you are downloading new MCM1 WUs, they come in one single MCM1_<WU#>_<WU#>.txt file with either 912 or 915 bytes, for each WU. Nothing else! I got several hundreds of those when I had to kick the tires on stuck downloads (actually, a couple of uploads) this morning. And this has been the very same every since MCM1 exists. Before and after the transition from IBM to Krembil. And on uploads, it is one single file less than 2KB in size, while the upload for MCM1 is one single file in the 9-10MB size range...

Ralf
[Nov 14, 2024 12:36:34 AM]   Link   Report threatening or abusive post: please login first  Go to top 
alanb1951
Veteran Cruncher
Joined: Jan 20, 2006
Post Count: 1317
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Unable to download tasks for a week

Ralf,

That IS the current MCM1 master file -- the "dead giveaway" is that if that is still waiting to download when all the MCM1 .txt files have downloaded no MCM1 tasks start until it has been received!

For some reason, MCM1 master files never seem to have the BOINC option to make them "sticky", so if a system runs out of MCM1 tasks the file gets deleted and has to be fetched again -- unfortunately, this tends to happen a lot when there are download problems.

As link64 said, the master file for SARC1 is a lot larger, but we're still back on Ovarian Cancer at the moment, and these tasks run a lot faster because the data set is smaller and easier to process...

I know I'm re-iterating what link64 said, but you apparently didn't believe it (or what Crystal Pellet said in the post before that); I hope the third time of telling you is the charm, but if you don't believe me either, find that file on one of your systems and delete it, then see what happens to any MCM1 tasks that might be present :-) [Hint: I don't think the client is smart enough to fetch it again once it thinks it has it...]

Cheers - Al.

P.S. -- it is not my intention to give offence, merely to help correct misunderstanding :-)

[Edited to note that Ovarian tasks run a lot faster than Sarcoma tasks...]
----------------------------------------
[Edit 2 times, last edit by alanb1951 at Nov 14, 2024 5:52:37 AM]
[Nov 14, 2024 5:41:29 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Crystal Pellet
Veteran Cruncher
Joined: May 21, 2008
Post Count: 1403
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Unable to download tasks for a week

Sorry, while there is A larger "master file" that gets downloaded if you have not crunched on MCM1 for a while, this is NOT THIS FILE.

Please show me a screenshot of two consecutive MCM1 downloads where this file is being download, the same one, twice (or multiple times) in a row!

When you are downloading new MCM1 WUs, they come in one single MCM1_<WU#>_<WU#>.txt file with either 912 or 915 bytes, for each WU. Nothing else! I got several hundreds of those when I had to kick the tires on stuck downloads (actually, a couple of uploads) this morning. And this has been the very same every since MCM1 exists. Before and after the transition from IBM to Krembil. And on uploads, it is one single file less than 2KB in size, while the upload for MCM1 is one single file in the 9-10MB size range...

Ralf

Ralf,

On this host I returned a last MCM1-task. So no WCG-tasks at all in progress, suspended, waiting or ready to start. WCG-queue empty.
After a while I requested new work. No dirty pictures, but the simple BOINC's Event Log:

World Community Grid	14 Nov 10:55:31	Finished upload of MCM1_0227711_8976_1_r286770252_0 (890 bytes)
World Community Grid 14 Nov 10:55:33 Sending scheduler request: To report completed tasks.
World Community Grid 14 Nov 10:55:33 Reporting 1 completed tasks
World Community Grid 14 Nov 10:55:33 Not requesting tasks: "no new tasks" requested via Manager
World Community Grid 14 Nov 10:55:36 Scheduler request completed

World Community Grid 14 Nov 11:15:08 Sending scheduler request: To fetch work.
World Community Grid 14 Nov 11:15:08 Requesting new tasks for CPU
World Community Grid 14 Nov 11:15:17 Scheduler request completed: got 1 new tasks
World Community Grid 14 Nov 11:15:17 Project requested delay of 121 seconds
World Community Grid 14 Nov 11:15:19 Started download of MCM1_0227703_9063_MCM1_0227703_9063.txt
World Community Grid 14 Nov 11:15:19 Started download of e55b6bdba4ed0b4b6e315c6767d68e3f.txt
World Community Grid 14 Nov 11:15:22 Finished download of MCM1_0227703_9063_MCM1_0227703_9063.txt (912 bytes)
World Community Grid 14 Nov 11:20:05 Finished download of e55b6bdba4ed0b4b6e315c6767d68e3f.txt (48177891 bytes)

If you're still not convinced, I can't help you.
----------------------------------------
[Edit 1 times, last edit by Crystal Pellet at Nov 14, 2024 10:31:20 AM]
[Nov 14, 2024 10:30:16 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Link64
Senior Cruncher
Joined: Feb 19, 2021
Post Count: 206
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Unable to download tasks for a week

I don't really understand, why anyone should run out of MCM work and generate unnecessary downloads on the already overloaded download server just to post a screenshot or log of a download of that file, when everyone crunching MCM can verify in few seconds, that this file is part of MCM and that it is used for every MCM WU, just check the entries for MCM WUs in the client_state.xml:

<workunit>
<name>MCM1_0227795_4376</name>
<app_name>mcm1</app_name>
<version_num>761</version_num>
<rsc_fpops_est>28148909341233.000000</rsc_fpops_est>
<rsc_fpops_bound>14074454670616500.000000</rsc_fpops_bound>
<rsc_memory_bound>419430400.000000</rsc_memory_bound>
<rsc_disk_bound>524288000.000000</rsc_disk_bound>
<command_line> -SettingsFile MCM1_0227795_4376.txt -DatabaseFile dataset-curatedOvarian_EarlyLate_v1.0.txt </command_line>
<file_ref>
<file_name>MCM1_0227795_4376_MCM1_0227795_4376.txt</file_name>
<open_name>MCM1_0227795_4376.txt</open_name>
</file_ref>
<file_ref>
<file_name>e55b6bdba4ed0b4b6e315c6767d68e3f.txt</file_name>
<open_name>dataset-curatedOvarian_EarlyLate_v1.0.txt</open_name>
</file_ref>
</workunit>

----------------------------------------

----------------------------------------
[Edit 2 times, last edit by Link64 at Nov 14, 2024 11:15:17 AM]
[Nov 14, 2024 10:58:48 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Crystal Pellet
Veteran Cruncher
Joined: May 21, 2008
Post Count: 1403
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Unable to download tasks for a week

I don't really understand, why anyone should run out of MCM work and generate unnecessary downloads on the already overloaded download server just to post a screenshot or log of a download of that file ...

Yeah, it is the best for the system, server and bandwidth when we all stop crunching ... or even posting and reading the forums.
[Nov 14, 2024 12:33:33 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 29   Pages: 3   [ Previous Page | 1 2 3 | Next Page ]
[ Jump to Last Post ]
Post new Thread