Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 22
Posts: 22   Pages: 3   [ 1 2 3 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 10286 times and has 21 replies Next Thread
QuantumEthos
Senior Cruncher
Joined: Jul 2, 2011
Post Count: 336
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
rose HPC Community Configuration RS - some advice

HPC Community Configuration RS - some advice

updated : http://science.n-helix.com/2018/09/hpc-pack-install-guide.html

HPC High performance computing clusters for supercomputer and web servers : Linux and windows compatible! - Improves efficiency of national compute resources

RS

https://docs.microsoft.com/en-gb/azure/virtua...igh-performance-computing

https://docs.microsoft.com/en-us/powershell/h...ng/overview?view=hpc16-ps

https://go.microsoft.com/fwlink/p/?linkid=837709

HPC Resources :

you will need a server certificate such as a comodo or digicert or microsoft > https://www.gogetssl.com/compare-domain-sslcertificates/

The certificate needs to be verifiable like digicert would be...

CCP_CONNECTIONSTRING=dn2.n-helix.com,dn.n-helix.com,worldcommunitygrid.org,lhcathome.cern.ch,cern.ch,boinc.bakerlab.org,8.8.4.4,localhost,nosh
ost,cloudflareworkers.com,cloudflare.com,34.225.23.59,52.42.7.116,208.67,cloud.sfr.fr,sfrcloud.sfr.f
r,webrlb01.cern.ch,boincai01.cern.ch

CCP_SCHEDULER=dn2.n-helix.com,dn.n-helix.com,worldcommunitygrid.org,lhcathome.cern.ch,cern.ch,boinc.bakerlab.org,8.8.4.4,localhost,nosh
ost,cloudflareworkers.com,cloudflare.com,34.225.23.59,52.42.7.116,208.67,cloud.sfr.fr,sfrcloud.sfr.f
r,webrlb01.cern.ch,boincai01.cern.ch

Rupert S

https://science.n-helix.com

https://www.microsoft.com/en-us/download/details.aspx?id=54507 Microsoft HPC Pack 2016 including linux

https://technet.microsoft.com/en-us/library/cc514029(v=ws.11).aspx all HPC Packs 2016,2012 to 2008 info and download

https://msdn.microsoft.com/en-us/library/ff976568.aspx Microsoft High Performance Computing for Developers - info and downloads

https://docs.microsoft.com/en-us/azure/virtua...-cluster-active-directory - information and virtualisation

https://docs.microsoft.com/en-us/azure/virtua...s/hpcpack-cluster-options

Azura win+linux+iso
https://docs.microsoft.com/en-gb/azure/virtua...igh-performance-computing

Instructions
https://docs.microsoft.com/en-us/powershell/h...ng/overview?view=hpc16-ps
https://docs.microsoft.com/en-us/azure/cloud-...up-hybrid-hpcpack-cluster

Netport Clarity HPC
https://docs.microsoft.com/en-us/powershell/h...-networking?view=hpc16-ps

https://docs.microsoft.com/en-us/powershell/h...-deployment?view=hpc16-ps

Appendix A: KMS Client Setup Keys
https://docs.microsoft.com/en-us/previous-ver...and-2012/jj612867(v=ws.11)

****
https://kubernetes.io/docs/concepts/
https://kubernetes.io/docs/setup/pick-right-solution/
https://kubernetes.io/docs/setup/pick-right-solution/#local-machine-solutions

docker pull evancui/hpcacm
docker pull kubernets/spinnaker-clouddriver
docker pull cloudnativelabs/kube-router
docker pull alpine
docker pull whatwedo/cloudflare-railgun
docker pull rungeict/cloudflare-railgun
docker pull rungeict/cloudflare-railgun
docker run --name railgun-memcached -d --restart=always memcached:latest
docker run --name=binddns-resolver -d -p 53:53/udp -p 53:53 -d -p 5353:5353/udp -p 5353:5353 -d --restart=always unixtastic/bind-dnssec-resolver

https://github.com/BOINC/boinc-client-docker
https://github.com/marius311/boinc-server-docker/blob/master/docs/cookbook.md
https://cosmicmar.com/2016/02/14/boinc-server-docker-1/
----------------------------------------
[Edit 7 times, last edit by QuantumEthos at Oct 2, 2018 8:20:25 PM]
[Jul 14, 2018 8:08:01 PM]   Link   Report threatening or abusive post: please login first  Go to top 
QuantumEthos
Senior Cruncher
Joined: Jul 2, 2011
Post Count: 336
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: HPC Community Configuration RS - some advice

Working :-)
[Sep 1, 2018 12:58:35 PM]   Link   Report threatening or abusive post: please login first  Go to top 
QuantumEthos
Senior Cruncher
Joined: Jul 2, 2011
Post Count: 336
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: HPC Community Configuration RS - some advice

[Sep 7, 2018 4:47:51 PM]   Link   Report threatening or abusive post: please login first  Go to top 
QuantumEthos
Senior Cruncher
Joined: Jul 2, 2011
Post Count: 336
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: HPC Community Configuration RS - some advice

at the bottom is a configuration that makes boinc installing on a cloud easily achievable,
You mentioned using a kubernet swarm in your 40 minute recording @ Boinc conference London https://drive.google.com/drive/folders/1cuuTnlGIMjKx9lbCHEAVFjvqjVdYWMCU

Also your other member mentions docker; The use of docker for boinc is an issue we are fighting for!

My personal contributions would be the suggestion of boinc detection of docker (docker launch takes 15 seconds plus so boinc would have to wait for the process to initiate in the windows task manager before..
The initial detection system would launch for Container VM; Of special note is the use of Linux containers; In particular Science Linux and Alpine (for size and usability) (science kernel version is possible)
docker pull alpine - check this out!

Kindly yours RS
[Sep 7, 2018 6:39:01 PM]   Link   Report threatening or abusive post: please login first  Go to top 
QuantumEthos
Senior Cruncher
Joined: Jul 2, 2011
Post Count: 336
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: HPC Community Configuration RS - some advice

Clarified : HPC Compute Network - Advise install - Windows & Linux Server by RS

Spread the word : Operation Liberate the Node - High performance computation for beneficial means of complimentary use to the future & survival.

Ways and means of obtaining scientific worth and usage though the usage of High Performance Computing or HPC that can indeed solve many of the humanitarian objectives of our worlds.

Boinc VM's are a specific objective of worth to the charitable science community and field research..

http://science.n-helix.com/2017/04/boinc.html

HPC Pack 2016 Update 1 fixes - install one after the other - after installing HPC Pack 2016 Update 1

https://www.microsoft.com/en-us/download/details.aspx?id=56964

https://www.microsoft.com/en-us/download/details.aspx?id=57174

https://www.microsoft.com/en-us/download/details.aspx?id=54507 Microsoft HPC Pack 2016 including linux

https://technet.microsoft.com/en-us/library/cc514029(v=ws.11).aspx all HPC Packs 2016,2012 to 2008 info and download

https://msdn.microsoft.com/en-us/library/ff976568.aspx Microsoft High Performance Computing for Developers - info and downloads

https://docs.microsoft.com/en-us/azure/virtua...-cluster-active-directory - information and virtualisation

https://docs.microsoft.com/en-us/azure/virtua...s/hpcpack-cluster-options

Azura win+linux+iso

https://docs.microsoft.com/en-gb/azure/virtua...igh-performance-computing

Instructions

https://docs.microsoft.com/en-us/powershell/h...ng/overview?view=hpc16-ps

https://docs.microsoft.com/en-us/azure/cloud-...up-hybrid-hpcpack-cluster

Netport Clarity HPC

https://docs.microsoft.com/en-us/powershell/h...-networking?view=hpc16-ps

https://docs.microsoft.com/en-us/powershell/h...-deployment?view=hpc16-ps

Appendix A: KMS Client Setup Keys

https://docs.microsoft.com/en-us/previous-ver...and-2012/jj612867(v=ws.11)

Operational achievement & field testing of HPC configuration.
You will need a server certificate such as a comodo or digicert or microsoft > https://www.gogetssl.com/compare-domain-sslcertificates/

Simple Final 4 - Test configuration - Operation : Help https://worldcommunitygrid.org , Boinc , Cloudflare , HPC & SFR SA .. Achieve dreams of computation that will exceed expectations of science today.

Install Microsoft HPC Pack 2016 or higher , performance HPC pro users ! including linux , for host settings of HPC Pack,

For CCP_CONNECTIONSTRING & CCP_SCHEDULER ; Set in the install setup when installed.
The node address you enter for HPC pack for the nodes, load the certificate into the next page of install & root certificate!

CCP_CONNECTIONSTRING=127.0.0.1,dn.n-helix.com,worldcommunitygrid.org,lhcathome.cern.ch,cern.ch,boinc.bakerlab.org,cloudflareworkers.com,
cloudflare.com,cloud.sfr.fr,sfrcloud.sfr.fr,webrlb01.cern.ch,boincai01.cern.ch

CCP_SCHEDULER=127.0.0.1,dn.n-helix.com,worldcommunitygrid.org,lhcathome.cern.ch,cern.ch,boinc.bakerlab.org,cloudflareworkers.com,
cloudflare.com,8.8.4.4,cloud.sfr.fr,sfrcloud.sfr.fr,webrlb01.cern.ch,boincai01.cern.ch

**** Some Alternative HPC Paths that compliment Microsoft HPC Pack

https://docs.docker.com/get-started/

Kubernets K8's Docker comes with a version for local clusters

https://kubernetes.io/docs/concepts/

https://kubernetes.io/docs/setup/pick-right-solution/

https://kubernetes.io/docs/setup/pick-right-solution/#local-machine-solutions

*docker containers of use* & BOINC Host/Client VM Cloud Configuration.
docker pull kubernets/spinnaker-clouddriver

docker pull cloudnativelabs/kube-router

docker pull alpine

docker pull whatwedo/cloudflare-railgun

docker pull rungeict/cloudflare-railgun

docker run --name railgun-memcached -d --restart=always memcached:latest

docker run --name=binddns-resolver -d -p 53:53/udp -p 53:53 -d -p 5353:5353/udp -p 5353:5353 -d --restart=always unixtastic/bind-dnssec-resolver

docker run -d --name=railgun -p 2408:2408 -e RG_ACT_TOKEN=ENTERTOKENHERE -e RG_ACT_HOST=127.0.0.0 -e RG_LOG_LEVEL=1 -e RG_WAN_PORT=2408 --link railgun-memcached:memcached --restart=always rungeict/cloudflare-railgun

https://github.com/BOINC/boinc-client-docker

https://github.com/marius311/boinc-server-docker/blob/master/docs/cookbook.md

https://cosmicmar.com/2016/02/14/boinc-server-docker-1/

Our suggestion of boinc detection of docker (docker launch takes 15 seconds plus so boinc would have to wait for the process to initiate in the windows task manager before..

The initial detection system would launch for Container VM; Of special note is the use of Linux containers; In particular Science Linux and Alpine (for size and usability) (science kernel version is possible)

docker pull alpine - check this out!

In other notes :
https://www.openfabrics.org/

http://www.opencompute.org/

https://openhpc.community/downloads/

https://azure.microsoft.com/en-us/resources/templates/slurm/

https://wikis.nyu.edu/display/NYUHPC/Slurm+Tutorial
https://wiki.hpc.uconn.edu/index.php/SLURM_Guide

Linux, Windows excetera HTCondor
https://research.cs.wisc.edu/htcondor/

Platform LSF will do much you can utilise, It runs on Windows. It is commercial, and can be purchased with support. https://www.ibm.com/it-infrastructure

If you are wondering what putty is in windows hpc pack,
Appearing to be SSH Shell client (console)

https://www.putty.org/

https://research.computing.yale.edu/support/h...ide/connect-windows-putty

http://inside.mines.edu/HPC-SSH-Putty

(C)RS
[Sep 8, 2018 9:15:21 PM]   Link   Report threatening or abusive post: please login first  Go to top 
QuantumEthos
Senior Cruncher
Joined: Jul 2, 2011
Post Count: 336
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: HPC Community Configuration RS - some advice

Our suggestion of boinc detection of docker (docker launch takes 15 seconds plus so boinc would have to wait for the process to initiate in the windows task manager before..

The initial detection system would launch for Container VM & also pull user data into the VM (the necessary ID from project add device & also set the VM and OS use for special case use and non generic project data,
Of special note is the use of Linux containers; This container potentially could batch multiple projects with finesse; In particular Science Linux and Alpine (for size and usability) (science kernel version is possible)

docker pull alpine - check this out!

Finally special contained secure environments for HPC Boinc Networking science is realised with relative technical aplume & thusly easily usable by the average; Non average user.

RS
----------------------------------------
[Edit 1 times, last edit by QuantumEthos at Sep 9, 2018 2:20:53 PM]
[Sep 9, 2018 2:18:22 PM]   Link   Report threatening or abusive post: please login first  Go to top 
QuantumEthos
Senior Cruncher
Joined: Jul 2, 2011
Post Count: 336
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: HPC Community Configuration RS - some advice

Simplified HPC Pack install guide : Science,Server & Web : Rupert S

http://science.n-helix.com/2018/09/hpc-pack-install-guide.html

https://www.microsoft.com/en-us/download/details.aspx?id=54507 Microsoft HPC Pack 2016 including linux


Windows server: Step 1

install Microsoft HPC Pack - Microsoft HPC Pack server/Workstation/Unmanaged server/node

Head Node

127.0.0.1,dn.n-helix.com,worldcommunitygrid.org,lhcathome.cern.ch,cern.ch,boinc.bakerlab.org,cloudflareworkers.com,
cloudflare.com,cloud.sfr.fr,sfrcloud.sfr.fr,webrlb01.cern.ch,boincai01.cern.ch

You will need a server certificate such as a comodo or digicert or microsoft.. for the next page of hpc pack install

https://www.gogetssl.com/compare-domain-sslcertificates/

Docker though the install service

powershell Install-PackageProvider -Name NuGet -Force
powershell Install-Module -Name DockerMsftProvider -Repository PSGallery -Force
powershell Install-Package -Name docker -ProviderName DockerMsftProvider -Force


Windows 7/8/10: Step 1

install Microsoft HPC Pack node or workstation/server

Head Node

127.0.0.1,dn.n-helix.com,worldcommunitygrid.org,lhcathome.cern.ch,cern.ch,boinc.bakerlab.org,cloudflareworkers.com,
cloudflare.com,cloud.sfr.fr,sfrcloud.sfr.fr,webrlb01.cern.ch,boincai01.cern.ch

You will need a server certificate such as a comodo or digicert or microsoft.. for the next page of hpc pack install

https://www.gogetssl.com/compare-domain-sslcertificates/

https://docker.com - docker CE for windows 7/8/10 install requiring download and install of docker community edition edge..

powershell Install-PackageProvider -Name NuGet -Force
powershell Install-Module -Name DockerMsftProvider -Repository PSGallery -Force
powershell Install-Package -Name docker -ProviderName DockerMsftProvider -Force

and then install docker CE

https://docker.com

Step 2 for both : Enable Kubernets in docker and set available CPU's & Resources then >

admin : CMD

docker run --name railgun-memcached -d --restart=always memcached:latest

docker run --name=binddns-resolver -d -p 53:53/udp -p 53:53 -d -p 5353:5353/udp -p 5353:5353 -d --restart=always unixtastic/bind-dnssec-resolver

Do not forget that the first DNS server is 127.0.0.1


selenium headless chrome (needs to be run again on boot atm)

docker run -it --rm -p=0.0.0.0:9222:9222 -p=0.0.0.0:443:443 --name=chrome-headless-selenium -v /tmp/chromedata/:/data selenium/standalone-chrome

port : 4444
[Sep 21, 2018 8:41:44 PM]   Link   Report threatening or abusive post: please login first  Go to top 
QuantumEthos
Senior Cruncher
Joined: Jul 2, 2011
Post Count: 336
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
rose Re: HPC Community Configuration RS - some advice

----------------------------------------
[Edit 1 times, last edit by QuantumEthos at Sep 22, 2018 6:57:38 PM]
[Sep 22, 2018 2:19:02 PM]   Link   Report threatening or abusive post: please login first  Go to top 
adriverhoef
Master Cruncher
The Netherlands
Joined: Apr 3, 2009
Post Count: 2346
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: HPC Community Configuration RS - some advice


Sorry, both are redirecting to SmartApe, Rupert.
[Sep 23, 2018 10:43:52 AM]   Link   Report threatening or abusive post: please login first  Go to top 
QuantumEthos
Senior Cruncher
Joined: Jul 2, 2011
Post Count: 336
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
rose Re: HPC Community Configuration RS - some advice

ah well

Verpus
https://api.goodservers.io/redirect/55182cf3569e880f0ad34e5a

Host1Plus South africa & World
https://api.goodservers.io/redirect/54fcc40a7b677c1624bc2043

Raba B on Sep 30, 3:31 PM

You can apply for our entrepreneurial program where you receive credits every month to use our services: https://www.softlayer.com/IBMGEP-application You can apply on the following link.

IBM Cloud
https://console.bluemix.net/catalog/
https://www.ibm.com/cloud/pricing
https://developer.ibm.com/academic/
----------------------------------------
[Edit 3 times, last edit by QuantumEthos at Sep 30, 2018 2:59:58 PM]
[Sep 23, 2018 7:21:23 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 22   Pages: 3   [ 1 2 3 | Next Page ]
[ Jump to Last Post ]
Post new Thread