Friday, September 21, 2018

HPC Pack install guide

HPC Pack install guide : Science,Server & Web : Rupert S

https://www.microsoft.com/en-us/download/details.aspx?id=54507 Microsoft HPC Pack 2016 including linux


Windows server: Step 1


install Microsoft HPC Pack - Microsoft HPC Pack server/Workstation/Unmanaged server/node

Head Node
127.0.0.1,dn.n-helix.com,worldcommunitygrid.org,lhcathome.cern.ch,cern.ch,boinc.bakerlab.org,cloudflareworkers.com,cloudflare.com,cloud.sfr.fr,sfrcloud.sfr.fr,webrlb01.cern.ch,boincai01.cern.ch

You will need a server certificate such as a comodo or digicert or microsoft.. for the next page of hpc pack install

https://www.gogetssl.com/compare-domain-sslcertificates/

Docker though the install service

powershell Install-PackageProvider -Name NuGet -Force
powershell Install-Module -Name DockerMsftProvider -Repository PSGallery -Force
powershell Install-Package -Name docker -ProviderName DockerMsftProvider -Force

Windows 7/8/10: Step 1


install Microsoft HPC Pack node or workstation/server

Head Node
127.0.0.1,dn.n-helix.com,worldcommunitygrid.org,lhcathome.cern.ch,cern.ch,boinc.bakerlab.org,cloudflareworkers.com,cloudflare.com,cloud.sfr.fr,sfrcloud.sfr.fr,webrlb01.cern.ch,boincai01.cern.ch

You will need a server certificate such as a comodo or digicert or microsoft.. for the next page of hpc pack install

https://www.gogetssl.com/compare-domain-sslcertificates/

https://docker.com - docker CE for windows 7/8/10 install requiring download and install of docker community edition edge..

powershell Install-PackageProvider -Name NuGet -Force
powershell Install-Module -Name DockerMsftProvider -Repository PSGallery -Force
powershell Install-Package -Name docker -ProviderName DockerMsftProvider -Force
and then install docker CE

https://docker.com

Step 2 for both : Enable Kubernets in docker and set available CPU's & Resources then >


admin : CMD

docker run --name railgun-memcached -d -p=127.0.0.1:11211:11211 -e https_proxy=http://127.0.0.1:3128 -e http_proxy=http://127.0.0.1:3128 -e ftp_proxy=http://127.0.0.1:3128 --restart=always memcached:latest

docker run -d --name=railgun -p 2408:2408 -e RG_ACT_TOKEN=ENTERapiTOKENHERE -e RG_ACT_HOST=127.0.0.1 -e RG_LOG_LEVEL=1 -e RG_WAN_PORT=2408 -e https_proxy=http://127.0.0.1:3128 -e http_proxy=http://127.0.0.1:3128 -e ftp_proxy=http://127.0.0.1:3128 --link railgun-memcached:memcached --restart=always rungeict/cloudflare-railgun

docker run -d --name=squid-memcached -p=127.0.0.1:3128:3128 -e RG_ACT_HOST=127.0.0.1 -e RG_LOG_LEVEL=1 -e https_proxy=http://127.0.0.1:3128 -e http_proxy=http://127.0.0.1:3128 -e ftp_proxy=http://127.0.0.1:3128 --link railgun-memcached:memcached --restart=always  minimum2scp/squid

docker run -d --name=nuster-memcache -p=127.0.0.1:8080:8080 -p=127.0.0.1:443:443 -p=127.0.0.1:80:80 -e RG_ACT_HOST=127.0.0.1 -e RG_LOG_LEVEL=1 -e https_proxy=http://127.0.0.1:3128 -e http_proxy=http://127.0.0.1:3128 -e ftp_proxy=http://127.0.0.1:3128 --restart=always nuster/nuster

docker run --name=binddns-resolver -d -p 53:53/udp -p 53:53 -d -p 5353:5353/udp -p 5353:5353 -e https_proxy=http://127.0.0.1:3128 -e http_proxy=http://127.0.0.1:3128 -e ftp_proxy=http://127.0.0.1:3128 -d --restart=always unixtastic/bind-dnssec-resolver

# Do not forget that the first DNS server is 127.0.0.1

docker run --name nax-waf -e BACKEND_IP=127.0.0.1 -e KIBANA_PASSWORD=Safe5 -d --link railgun-memcached:memcached --restart=always scollazo/naxsi-waf-with-ui

docker exec squid-memcached curl http://127.0.0.1/

****

Selenium runs once - you need to re run it on boot

https://store.docker.com/community/images/selenium/standalone-chrome
https://github.com/SeleniumHQ/docker-selenium

docker run -d -it --rm -p=0.0.0.0:9222:9222 -p=0.0.0.0:443:443  --link railgun-memcached:memcached --name=chrome-headless-selenium -v /tmp/chromedata/:/data selenium/standalone-chrome

port : 4444


****
*waf firewall*
https://hub.docker.com/r/scollazo/naxsi-waf-with-ui/
docker pull scollazo/naxsi-waf-with-ui

*zend php stack is licenced*
https://hub.docker.com/_/php-zendserver/
docker pull php-zendserver

*nginx php stack*
https://hub.docker.com/r/richarvey/nginx-php-fpm/
docker pull richarvey/nginx-php-fpm

*node js*
https://hub.docker.com/r/mhart/alpine-node/
docker pull mhart/alpine-node

*squid cache*
https://hub.docker.com/r/minimum2scp/squid/
docker pull minimum2scp/squid

*Nuster http Cache - needs configuration !*
https://github.com/jiangwenyuan/nuster
https://hub.docker.com/r/nuster/nuster/
docker pull nuster/nuster

*railgun*
https://www.cloudflare.com/docs/railgun/installation.html
https://www.cloudflare.com/docs/railgun/daemon_execution.html

https://hub.docker.com/r/rungeict/cloudflare-railgun/
docker pull rungeict/cloudflare-railgun

https://www.cloudflare.com/website-optimization/railgun/?r=1

****
dhcp servers

freeradius dhcp servers
https://hub.docker.com/r/freeradius/freeradius-server/
docker pull freeradius/freeradius-server
docker run -d --name my-radius -p 1812-1813:1812-1813/udp --restart=always freeradius/freeradius-server

RackHD dhcp server
https://rackhd.readthedocs.io/en/latest/
https://github.com/RackHD/
https://hub.docker.com/r/rackhd/isc-dhcp-server/
docker pull rackhd/isc-dhcp-server

docker run -d --name rackhd-dhcp -p 1812-1813:1812-1813/udp --restart=always rackhd/isc-dhcp-server

Alpine dhcp
https://hub.docker.com/r/joebiellik/dhcpd/
docker pull joebiellik/dhcpd
****

https://hackernoon.com/a-kubernetes-guide-for-docker-swarm-users-c14c8aa266cc

https://blog.alexellis.io/you-need-to-know-kubernetes-and-swarm/

Further reading for swarm node configuration

https://github.com/docker/labs/blob/master/swarm-mode/beginner-tutorial/README.md

Traefik : Swarm cloud single ip/site url; header wall that keeps all nodes hidden within to handle web traffic from the singular web portal to inside the node array list (funnels Data, Hence > traefik)
https://github.com/bee42/traefik-with-docker
https://github.com/bee42/docker-orchestration-swarming

https://hub.docker.com/r/bee42/docker-orchestration-swarming/
docker pull bee42/docker-orchestration-swarming

****
Open science grid including tensor flow & GPU
https://hub.docker.com/u/opensciencegrid/

swarm elasticsearch
https://hub.docker.com/r/agoryachev/docker-swarm-elasticsearch/

****
Media server
https://hub.docker.com/r/kurento/kurento-media-server/
docker pull kurento/kurento-media-server
docker run -d --name kms-mediaserver -p 8888:8888 kurento/kurento-media-server

Image neural enhancer
https://github.com/alexjc/neural-enhance
https://hub.docker.com/r/alexjc/neural-enhance/
docker pull alexjc/neural-enhance

caffe gpu
https://hub.docker.com/r/tleyden5iwx/caffe-gpu-master/
docker pull tleyden5iwx/caffe-gpu-master
****

http://science.n-helix.com/2018/09/hpc.html - a bit about boinc in terms of Docker & Microsoft HPC Pack
http://science.n-helix.com/2017/04/boinc.html - HPC, Optimisation & Boinc - programming

No comments: