| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
| World Community Grid Forums
|
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 6
|
|
| Author |
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
In December last year I decided to assemble my own dedicated little crunching machine. It had to fulfill certain criteria regarding power usage, noise level, and size within a limited budget. As a result of those criteria, I decided to go without an HDD and bought a cheap USB stick with USB 3.0 connectivity. Unfortunately Microsoft does not allow yet to install Windows on a USB stick so I had to go with Linux instead. As I also wanted to crunch CEP2 as well, although not exclusively, I somehow had to manage the disk writes to avoid trashing the new USB stick in no time. Because there are only limited capabilities to reduce disk writes through the BOINC software itself, I decided to use a RAM disk for the BOINC data directory instead.
----------------------------------------So here comes an attempt in detailing what I did to create a RAM disk, to change the BOINC directory, to set up an backup routine and to get the RAM disk correctly started and stopped during the start-up and shutdown procedures of Linux. There are many ways to do it, this is just one way, I guess. Starting point in my case: - running Lubuntu 11.10 64-bit (any other Ubuntu derivative will do) properly installed on an USB stick (no Live mode) - working and running BOINC 6.12.33 installed through Ubuntu packages - an open terminal (CRTL+ALT+T) - a healthy curiosity regarding linux and terminal commands Create RAM disk and change data directory 1. Stop BOINC-client from running. sudo service boinc-client stop 2. Create a directory as a mounting point for the RAM disk. (Note: The default BOINC data directory was /var/lib/boinc-client ) sudo mkdir /var/lib/boinc-client-RAM 3. Mount a temporary file system (RAM disk) with the aforementioned directory as mounting point and check if you succeeded. (Note: tmpfs is used over ramfs because it allows to set a fixed size of RAM and uses the swap file as safety. Use whatever size you want. In my case it was 65% of the RAM but absolute terms like 2000M are also possible.) mount -t tmpfs -o size=65%,noatime none /var/lib/boinc-client-RAM 4. Copy data from current BOINC data directory into the new RAM disk data directory and check if you succeeded. sudo cp -r /var/lib/boinc-client /var/lib/boinc-client-RAM 5. Change the owner of those files back to "boinc", remove writing rights for group and others and check if you succeeded. (Note: The reason behind this is to 100 % mirror the original data-directory. Not sure if it is necessary, though.) sudo chown -R boinc:boinc /var/lib/boinc-client-RAM 6. Set-up Linux to create the RAM disk on start-up. You need to add a line to the file "fstab" in /etc/. (Note: Change the line accordingly to what you have used in 3., don't forget to save with CRTL+O, and close with CRTL+X) sudo nano /etc/fstab 7. Change the BOINC data directory to the new directory of the RAM disk. There are two files that need to be configured accordingly. (Note: I am using my mounting point for the RAM disk, if you change that to another directory you have to change the lines accordingly.) sudo nano /etc/default/boinc-client sudo nano /etc/init.d/boinc-client 8. Start the BOINC-client and check within BOINC-Manager through the messages function if the new data directory is used. It will say so in the beginning of the messages, approximately the 7th line which directory is used. sudo service boinc-client start Initialize/Terminate data directory during start-up/shutdown As of now, if you restart your computer everything in the RAM disk will be lost and the BOINC-client will not start as it cannot find the files in the data directory, which of course is empty. So we need to create a backup-directory which will be used during shutdown to write everything from the RAM disk to the HDD (USB stick) and back at start-up. 9. Create backup directory and change the owner to "boinc". sudo mkdir /var/lib/boinc-client-Backup 10. Change the boinc-client service, so that it will copy everything to HDD when stopped and back to the RAM disk when started. For this we need to change the file "boinc-client" in /etc/init.d/ (Note: The boinc-client should be running, so that everything will be copied into the backup directory during the stop process, else the current RAM disk will be overwritten when the boinc-client service is started with the still empty backup directory. As an alternative you can make a manually backup by copying everything from the data directory into the backup directory.) sudo nano /etc/init.d/boinc-client Automatic Backup As of now, every time someone stops or starts the service "boinc-client" the content of the RAM disk will be copied to the HDD (USB stick) or to the RAM disk. This happens automatically if you start or shutdown your linux and manually if you use "sudo service boinc-client start/stop". The only thing missing is an automatic backup after every X hours in case of an power/system failure. For this I use crontab. 11. Create a task that will take a backup every X hours, using "crontab". (Note: In my case, it will only take a backup at 6 o'clock in the morning everyday to minimize disk usage on the usb stick. Every two hours for example would be 0 */2 * * * /usr/... Furthermore it is mandatory to finish the last line with a RETURN key, especially if only one line is inserted, to create a blank line at the end of the file. Don't use blank lines between job entries, only for the last job. ) sudo crontab -e 12. To minimize the probability of losing finished WUs which have not yet reported to WCG, BOINC can be adjusted to report immediately by changing the cc_config.xml file. (Note: Having BOINC connect every hour might work as well, but I have not tested it.) sudo nano /etc/boinc-client/cc_config.xml Note That should be everything, I hope. Be careful, double check and experiment with the code. I cannot guarantee that everything is spelled correctly, done in the right order or even explained correctly. A native speaker will have surely realized that I am not one of them, although I have tried to be as accurate as possible, I never will be. [Edit 2 times, last edit by Former Member at Mar 3, 2012 9:26:14 PM] |
||
|
|
Rickjb
Veteran Cruncher Australia Joined: Sep 17, 2006 Post Count: 666 Status: Offline Project Badges:
|
On behalf of other WCG members, thanks for your contribution on this.
I don't think I'll be using a RAM disc, but one can never be sure. There have been other discussions on RAMdiscs for WCG, eg in the CEP2 forum at RAMDisk Most info there relates to running under Windows, and the thread is fairly stale, but I hope my link is useful. [BTW, your English seems perfect to me, and far better than that of most of us Australian "natives"]. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Thank you so much too, kd1983.
If you don't mind I'll create a RAMDisk topic in the Start Here FAQ index with link to your post. If anyone has comments or need for clarification, would you be so kind to edit the OP. Could follow it from top to bottom, maybe a little spacing for the second half of the instructions steps under point 10 Great job and thanks for sharing. --//-- |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
I appreciate the compliments. I wish I had learned earlier on how to enlarge the edit box, as it was really burdensome to have so little space at hand for the original post.
@SekeRob Of course, I will allow it. I edited the post according your suggestions and put it on my watch list. Sadly there is no apparent indication what those different watch type settings will do, so I left it on System Default. Hopefully it will give me an email notice, but time will tell. |
||
|
|
OldChap
Veteran Cruncher UK Joined: Jun 5, 2009 Post Count: 978 Status: Offline Project Badges:
|
This is Really appreciated, by me at least. It contains so much of what I have been searching for. Thank you.
----------------------------------------Leaves me wondering how much ram I would need to have installed to run the whole WCG/Boinc process and whether or not that would produce any tangible gains. ![]() |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Well, performance-wise I expect an improvement in efficiency. If this improvement is indeed tangible will highly depend on what was used as an alternative before. If this alternative is an USB stick it will result in a more tangible improvement than someone who would use a SSD otherwise.
----------------------------------------Looking at my efficiency values through BoincTask, I see efficiencies of around 99.2 to 99.8 % with any non CEP2 project and 97-98 % with CEP2 on all four cores. Efficiency is still fluctuating around checkpoints, though. Sadly I have no comparable results from before, so it is just a guessing from my part wherever or not the improvement was tangible. Another valuation could be done through measuring the uptime of a system, which should be improved by a likely lesser probability of HDD failures but bought with a longer downtime in cases of power or system failures through recalculating those WUs that were lost in the RAM. Difficult to say what will dominate, as it is highly theoretically and would depend on what type of HDD one is using and of course how stable the power and system is in the long run. Of course, if you have no extra HDD lying around -- as it was in my case -- the spending of money should also be considered. An HDD would have cost me at least twice to triple to that of a good USB stick. It hopefully will last as long as an HDD and uses less power. On the other hand I would most likely not have bought extra RAM, so it is still wishy-washy in the end. OP Edit Note I edited the OP to reflect some problems with crontab in case of a missing blank line at the end of the crontab file. Hope that will spare others some time. [Edit 2 times, last edit by Former Member at Mar 7, 2012 1:01:38 PM] |
||
|
|
|