Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 822
Posts: 822   Pages: 83   [ Previous Page | 55 56 57 58 59 60 61 62 63 64 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 1870130 times and has 821 replies Next Thread
Jim1348
Veteran Cruncher
USA
Joined: Jul 13, 2009
Post Count: 1066
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work unit availability

Surely, there must be technical tricks to tackle that write tsnumami...

If you are on Windows, use PrimoCache (use the write-cache only):
https://www.romexsoftware.com/en-us/primo-cache/index.html

If you are on Linux, use their built-in write cache:
https://lonesysadmin.net/2013/12/22/better-li...rformance-vm-dirty_ratio/

I have not bothered to looks at the writes here, since they aren't sending out many, but I would think that a cache size of 2 GB and a latency (write-delay) of 30 minutes should work. I use them on all of my machines to protect the SSDs.
[Jun 8, 2021 1:05:47 PM]   Link   Report threatening or abusive post: please login first  Go to top 
nanoprobe
Master Cruncher
Classified
Joined: Aug 29, 2008
Post Count: 2998
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work unit availability

Surely, there must be technical tricks to tackle that write tsnumami...

If you are on Windows, use PrimoCache (use the write-cache only):
https://www.romexsoftware.com/en-us/primo-cache/index.html

If you are on Linux, use their built-in write cache:
https://lonesysadmin.net/2013/12/22/better-li...rformance-vm-dirty_ratio/

I have not bothered to looks at the writes here, since they aren't sending out many, but I would think that a cache size of 2 GB and a latency (write-delay) of 30 minutes should work. I use them on all of my machines to protect the SSDs.

I assume you're using the paid version on windows? I ask because the the only delayed write settings I could find were 10,60 and 300 seconds or infinite.
----------------------------------------
In 1969 I took an oath to defend and protect the U S Constitution against all enemies, both foreign and Domestic. There was no expiration date.


----------------------------------------
[Edit 1 times, last edit by nanoprobe at Jun 8, 2021 6:13:13 PM]
[Jun 8, 2021 5:48:07 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Jim1348
Veteran Cruncher
USA
Joined: Jul 13, 2009
Post Count: 1066
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work unit availability

I assume you're using the paid version on windows? I ask because the the only delayed write settings I could find were 10,60 and 300 seconds or infinite.

Yes. I don't know of any free ones. Primo spent years developing it to eliminate all the errors. I have never seen such a long beta.

You could try a ramdisk instead, but you have to size them to hold the entire BOINC data folder. I think there are some free ones, but I use Primo Ramdisk (the best monitoring tools) or Dataram.
----------------------------------------
[Edit 1 times, last edit by Jim1348 at Jun 8, 2021 7:28:35 PM]
[Jun 8, 2021 7:25:34 PM]   Link   Report threatening or abusive post: please login first  Go to top 
erich56
Senior Cruncher
Austria
Joined: Feb 24, 2007
Post Count: 300
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work unit availability

You could try a ramdisk instead, but you have to size them to hold the entire BOINC data folder. I think there are some free ones, but I use Primo Ramdisk (the best monitoring tools) or Dataram.

I am using the Primo Ramdisk Professional version, which provides a Ramdisk up to 32GB (the free version has only 16GB, as far as I remember). Price was USD 49.
My total RAM is 64GB, so I used 32GB for the RAMdisk (good part of it is being used by the ATLAS tasks from LHC, which are also known as SSD killers).
Everything works perfect and very fast.
[Jun 9, 2021 2:10:16 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Jim1348
Veteran Cruncher
USA
Joined: Jul 13, 2009
Post Count: 1066
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work unit availability

My total RAM is 64GB, so I used 32GB for the RAMdisk (good part of it is being used by the ATLAS tasks from LHC, which are also known as SSD killers).
Everything works perfect and very fast.

That reminds me of the good old days, when I had a large RAMdisk for the CEP2 project here on WCG. It had a horrendously high write rate, over 1 TB/day for 8 cores. Also, CPDN had a high write rate back then, though less now.

The limitations on the write rate for mechanical disks caused errors too. I was able to eliminate them with the ramdisk. With enough memory, you can do a lot of things.
[Jun 9, 2021 3:51:36 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Grumpy Swede
Master Cruncher
Svíþjóð
Joined: Apr 10, 2020
Post Count: 2515
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work unit availability

Well, I haven't seen any GPU work for days now. Nothing for Nvidia, and nothing for my iGPU.
[Jun 10, 2021 2:36:43 PM]   Link   Report threatening or abusive post: please login first  Go to top 
kittyman
Advanced Cruncher
Joined: May 14, 2020
Post Count: 140
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work unit availability

The kitties' kibble bowl is bare here as well. Nothing coming through today.

Meowsigh

EDIT..........
I guess all it took was for the kitties to meow louder...........LOL.
After 7-1/2 hours of nothing, I posted this and got 16 fresh bits of kibble half an hour later.

Meow!
----------------------------------------

----------------------------------------
[Edit 1 times, last edit by kittyman at Jun 10, 2021 3:10:33 PM]
[Jun 10, 2021 2:42:41 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Grumpy Swede
Master Cruncher
Svíþjóð
Joined: Apr 10, 2020
Post Count: 2515
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work unit availability

Hmm, I got one, OPNG_0050453_00068_0 smile
[Jun 10, 2021 3:21:45 PM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work unit availability

A quick note about workunit availability for the project. As noted in the post about the outcome of the stress test the researchers have a limit on how fast they can process results and make use of them. We are currently working with them to target about 500 batches per day (this is combined between CPU and GPU). It is a little tricky to manage to this because even though there is a consistent number of ligands per batch, the number of workunits created per batch varies based on the estimated difficulty of the ligands. Thus we have to tweak the rate we send out workunits.

Right now the 7 day average of batches for GPU is 727 batches/day and CPU is 185 batches/day. This is above the 500 batches/day goal. As a result, we reduced the GPU pace 2 days ago from 3000 workunits/day to 1700 workunits/day. We have been attempting to reduce the pace of the CPU work as well. Unfortunately, there is a fairly large set of users who only have OpenPandemics selected and thus we are not able to shift as much of the CPU power to our other projects as I had hoped. We will continue to adjust things where we can to support the 500/batches per day goal and to shift work to GPU as much as possible without making the CPU work intermittent.

I would ask that if you are someone who has only OpenPandemics selected, please consider adding Mapping Cancer Markers and/or the African Rainfall Project in addition to OpenPandemics. OpenPandemics will continue at the same pace regardless of how much CPU power is allocated to it as the GPU side will pick up any slack left over from the CPU side.
[Jun 10, 2021 3:26:41 PM]   Link   Report threatening or abusive post: please login first  Go to top 
kittyman
Advanced Cruncher
Joined: May 14, 2020
Post Count: 140
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Work unit availability

"As a result, we reduced the GPU pace 2 days ago from 3000 workunits/day to 1700 workunits/day."

Well, it's no wonder that the kitties have been going hungry.......
Guess that's not gonna improve at this rate.

Meowsigh
----------------------------------------

[Jun 10, 2021 3:55:24 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 822   Pages: 83   [ Previous Page | 55 56 57 58 59 60 61 62 63 64 | Next Page ]
[ Jump to Last Post ]
Post new Thread