Friday, July 12, 2019

WCG - Boinc - Community Science Proposals

WCG - World Community Grid proposed topics of interest

Boinc : Bare Metal VM

Firstly the world community at WCG propose to do many a worldly good for the aiding of social order & good health..

List:

Study into the fungle foot infections including bovine fungle foot rot,
While this topic may seem minor the effects on society of bad feet is annoying and clinically depressing to so many.

Study of the various effects of tar including examination and computation of the locations and formation of tar & excessive poisons in the lungs,
Proposal includes all collected data & informed research in biomedical databases.

Healthy living is the requirement of mankind & alien species alike.

Study of bone fractures & the auxiliary study of XRay examination with radiation dosage assessments., Further study will improve on already successful research at WGC,WHO & MSF (UN)

Study of chemotherapy & radio interactions with antibody proposals for medical grade chemical usage minimisation & optimisation.

Study on the effects of skin burns on healing & of the environment-s best suited to self healing bodies; Under therapy & or ointment treatments.

https://www.worldcommunitygrid.org/about_us/viewAboutUs.do

https://www.worldcommunitygrid.org/research/viewSubmitAProposal.do

Rupert S

Tuesday, June 18, 2019

Vulkan Kernel - Windows - Linux - Android - Apple

The Vulkan Kernel - Windows - Linux - Android - Apple

References to VULKAN can be replaced or supplemented with Metal (Mac OS) & Direct X

All driver parameters are set according to the Vulkan principle with hardware and AVX & Float versioning...

Under the level 2 kernel api calls, Direct messages to the driver modules in Vulkan compatible script, The kernel is fittingly representing the screen resolution though mode & VESA Standard 2D & 3D instructions..

Mode query will one proposes include Vulkan API call queries to all compatible devices..
For this purpose the api will call an ID on the device for driver model/Version.

Once the VESA & Vulkan standards are found; Mode setting can be carried out by the OS,
However the Open CL & GL functions for screen settings can be used pure.

Depending on space in the bios uefi the calls may have already been made to call the screen optimum into being, Bios calls can reference hardware, IO, DMA & Memory/Feature Sets,
Such settings are equivalent to environmental flags (Set).

Pure Vulkan kernel calls in Kernel speed up the process & lower RAM usage,
Possibly increase security (ideally)

Saturday, June 8, 2019

Website Development : 3D : Vulkan Stack for web page,data,science & gaming.

Website Development : 3D : Vulkan Stack for web page,data,science & gaming.


Stack list:

HTML DOM (Document Object Model) >
PHP & Database >
HTML5 >
JS - JQuery & JQuery 3D feature stack & jquery ui >
WebGL >
WebGL Compute &
Streamline API
Vulkan API - Direct Render : Ray trace the Audio/Video/Text & Visuals and force/ gravitational effect paths,

Use Vulkan - Direct Compute OpenCL to trace , optimisation and pre-render Vertices,Borders,Renderings & pass Vector trace to GPU & Back To Direct Compute OpenCL.

= Interactive 3D & Web

https://science.n-helix.com/2017/02/open-gaming.html

https://science.n-helix.com/2016/04/3d-desktop-virtualization.html
***

HTML DOM (Document Object Model)

"The DOM is the structure a web browser generates from an HTML file. The browser reads the HTML file and generates a version of the elements that is formatted for your JavaScript code to communicate with. We need this “translated” version of the HTML so that we can use JavaScript to talk to the elements on the page. If JavaScript could not talk to the DOM, we wouldn’t be able to use JavaScript to change the appearance of the page."

https://generalassemb.ly/coding/full-stack-web-development/dom-manipulation

https://www.w3schools.com/js/js_htmldom.asp

3D Document manipulation though the use of JQuiry

***

PHP & Database stack to handle program data and formatting...
& because PHP can output pages and windows in code formatted in DOM and HTML5
We can create menus and pages that do not require the use of large file sets..

Code repetition is the secret of the PHP and database system,
Other systems like sites written in HTML require pages to be written to the server &

PHP with the database stack is a sensible system to reduce the nessety to use Lots of storage for the site code,

PHP code can create multiple window sets with Dom orientated Dynamic JS and HTML5 Dynamically
Using the principle of JQuery & JQuery 3D feature stack.

****

So HTML5 & DOM create windows, frames & interactive content:

However we need to use standards that create animated objects & people,
While we can draw basic 3D objects in HTML5 we also would like them animated..

3DMax, AutoCAD & other formats provide motion for vectors so we need HTML5 to animate objects though OpenGL, WebGL & Vulkan.. So we need formatting to create motion in GL.

Tables of motion & reaction, created in small database sets, Database sets can be compressed & should be.

For this purpose we use databases, more than 1 so we are able to store sets of objects within the scene or web page.. In essence we stack compressed databases & this allows us to both interact with the page & stop downloading data or save bandwidth.

****

Trace & Compute : Open CL Direct Compute, GLES,GL,Vulkan,Direct X : Video Effects
Ray trace the audio and force/ gravitational effect paths
Use open CL to trace , optimisation and pre-render, Light & Sound & effects such as force-fields,

By Intercepting occlusion in comparison to OpenCL Direct compute directives of force & motion & energy .. Direct compute (OpenGLES 3.1,Vulkan, Open GL & Direct X..

Direct Compute Open CL is able to ray trace anything from simple dynamic effects to bullet trace sound effects, With direct mapped effective & efficient Direct Compute OpenCL in 3 modes:

direct mapped effective & efficient Direct Compute OpenCL in 3 modes:
Real time
Pre-Rendered on load (scene & lighting, base shadow effects)
Interception real-time pre-render (Microseconds) with Spontaneous : Active CL (tm)

Use of cone, AE Effects lower the CPU/AVX/GPU processor usage while maintaining effectiveness.

Library builds reduce development costs with Real-Time Engines.

Other effective use of compute such as maths are 100% Effective both in games & on the web.

We can use CPU & GPU MipMap & Tessellation  RiS with micro smoothing predictive tessellation with map fonts, We can also do colour maps and lut conversion for dynamic contrast & Sound for the Realtek Audio codec! We can do this for video also...

Creating SDK ARM/X86/GPU/FPMG/SiMD

(c)RS
****

LUT tables and tone mapping: Vectors
https://gpuopen.com/using-amd-freesync-2-hdr-gamut-mapping/

On the subject of LUT tables and tone mapping, 2 methods are available to us..

The Vectors can be mapped RT with ray tracing (they work out the vector)

The Vectors and dimensions can also be worked out with Open CL and Direct Compute..
Both OpenGL/Vulkan & Direct X have direct compute..

Many forms of vector calculation that involve intricate maths  can be worked out in vector or OpenCL Vector library function, The advantage of Open CL Libraries are that functions and tables can be worked out without ever having to re program the maths solving OpenCL Code,

Such that Open CL & Direct compute libraries can for-fill many tasks, Bearing in mind that Open CL & Direct compute are work solve time controlled we are able to use the functions for many tasks including web browser maths and composure, With these examples we' will define the future of display maths code & logic.

AVX & Float can obviously be used leaving Compute vectors like SIMD viable for code logic.
Compute Shaders are also able, Long logic denotes the advantage of Vectored OpenCL & Direct compute/AVX.

Vectored code : tessellation & other functions using SIMD & Compute Shader maths:
https://www.youtube.com/watch?v=0DLOJPSxJEg

VR-VMP-3D - Vector tables/SIMD/RayTracing/High Precision Float & Light/Shade & Colour HDR Mapping & Polymorphic HDR 3D Sound, Texture emulation of feel, touch and sensation though Direct Compute Shaders & poly numeric maths.

Haptic 3D feeling/Sensation/Visuals/Sound & Audio for JS/script & code/Open CL/Direct Compute for 3D/Video/Internet HPC.

Sensational Virtual 3D Web/Video/Classic Video/Games/Audio/Fonts with haptic sensation and touch! All new JS ML code to make true sensation : real feels for emotional highs as you chat, tip or cam your game experience & do research high performance compute.

https://science.n-helix.com/2014/08/turning-classic-film-into-3d-footage-crs.html

https://science.n-helix.com/2019/06/vulkan-stack.html

(c)Rupert S

****
Networking, Audio & Display Codecs:
Have you thought about using shaders in Networking ? to realise the network data strategy...
The same is true for displays & Audio & other Science data such as Neural networks,
Image improvement and encoding & entertainment video codecs, 64Bit HDR Dynamic Contrast

https://is.gd/SuggestedShaders

https://is.gd/ShadersSDK

https://is.gd/CodecShaders

https://is.gd/InterpolationShaders

We can apply the interpolation to video for smoothing and vectorisation of the video elements in float for sharpening & to our interpolation for tessellation of the RiS sharpening for all our GPU and CPU elements.

(c)RS

https://reshade.me/

https://reshade.me/presets

****
https://science.n-helix.com/2012/09/geometric-calculating-machines.html
WebCLGL : Libraries & JS
https://github.com/stormcolor/webclgl
https://github.com/stormcolor/webclgl/blob/master/dist/webclgl/WebCLGL.min.js

WebCLGL use WebGL2 specification to interpret code.
WebGL is used like OpenCL for GPGPU calculus using the traditional Render To Texture technique.

****
WebGL Compute
https://www.khronos.org/registry/webgl/specs/latest/2.0-compute/#diff-with-gles31

https://www.khronos.org/assets/uploads/developers/library/2017-webgl-webinar/Khronos-Webinar-WebGL-20-is-here_What-you-need-to-know_Apr17.pdf

****
ROCm & Vulkan Drivers : Debian/Ubuntu Linux install :

The thing with AMD drivers is that you need to uninstall the previous driver completely first before installing the new one.. ROCm sounds promisingly likely to improve with the laboratories promising to improve ROCm with cray & does not require uninstalling ...

https://www.phoronix.com/scan.php?page=news_item&px=Radeon-ROCm-2.3-Released

https://rocm.github.io/blog.html

https://github.com/RadeonOpenCompute/ROCm

ROCm & Vulkan Drivers install

run this after downloading file (google drive): https://is.gd/Install_gpl_ROCm_amd_drv_sh

sudo chmod 774 Install-gpl-ROCm-amd-drivers.sh
sudo ./Install-gpl-ROCm-amd-drivers.sh

****
Open Source Driver for Vulkan : Debian/Ubuntu/Linux
https://github.com/GPUOpen-Drivers/AMDVLK

run this after downloading file: https://is.gd/Install_gpl_amd_drivers_sh

sudo chmod 774 Install-gpl-amd-drivers.sh
sudo ./Install-gpl-amd-drivers.sh

****
GL to Vulkan : gfx-portability : Prototype library implementing Vulkan Portability Initiative using gfx-hal. See gfx-rs meta issue for backend limitations and further details.

https://github.com/gfx-rs/portability

****
OpenCL/OpenGL/Vulkan API : Mac:Windows:Linux:Android

https://github.com/KhronosGroup/MoltenVK/releases/tag/v1.0.35

https://github.com/KhronosGroup/OpenCL-ICD-Loader

https://github.com/KhronosGroup/OpenCL-CLHPP

****
Texture & polygon optimiser & compressor

https://github.com/GPUOpen-Tools/Compressonator/releases

https://github.com/KhronosGroup/glTF-Compressonator

****
Speeding up websites JS - for JQuery, PHP excettera! Very exciting for app development & Boinc SDK

Fetch code includes optimisation - to be run before JQuery

https://is.gd/FetchB4JQuery

Require-min to be run before JQuery - migration is for older version compatibility

Site Efficiency!

https://requirejs.org/docs/download.html

https://jquery.com/download/

https://jqueryui.com/

(c)RS

Sunday, May 26, 2019

Compiler Optimisation : CPU/GPU/Vector/Float : Transparent Open CL Direct compute

Compiler Optimisation : CPU/GPU/Vector/Float : Transparent Open CL Direct compute.

OpenCL, Web CL, Web Compute, Direct compute & Integration of CL code into gaming & web content..
We are able to utilise processing unite types by running the majority of code within a single class,
Such is the case, But we need to optimise the interruption cycle with Open CL Compute,

After all is it not the objective to receive code that compiles well on all?

Proposing to streamline the coding stacks into the Open CL pipeline allows simplification of the complicated task of writing in Open CL & also optimises code is such a way that it is :

Faster, Smaller & safer.

****

To implement the usable and fully functioning implementation of Open CL direct compute,
For the improvement of CPU MMX,scalar,Vector pipelines into the system architecture of future & present systems.

The open CL, Direct compute pipeline is predominantly vector & therefore,
Open CL Direct compute pipelines are by nature compatible with Scalar Vector architecture..
This means CPU,Float,Vector & GPU, This also means that Open CL structures can be directly encoded into scalar vector,GPU & CPU pipelines easily using vectorised compilers,

The advantage of vector pipelines is the CACHE; By this i mean all processor caches,
In type vectorised pipelines are also faster in integer.

While directly vectoring pipelines may be problematic, Mathematically clean balanced vectors are by nature error proof & fast.

Therefore OpenCL Direct compute is implementable by encoding that is to say compiling directly..
to the CPU feature sets & GPU, Vectorised floats are manageable in AVX,Scalar Vector functions and usable by floating point units, non floating point variables are capable of CPU integer encode.

Compiling code directly from languages in GCC & other compilers into Vectorisable variable adaptable code is also manageable in transparent compilation that makes invisible the necessity to compile code that is only usable one single way,

With pages in php with database backing rendered Though:
OpenCL Direct Compute maths scripting the output into Vulkan,ESGL & DirectX..

The usability of Open CL compilers; Enabled to write instructions for Vector processing units & float & Yes even Integer instruction sets such as X64, Is most important.

Function of the Floating unit : FPU : Scalar Vector Unit & Integer ..
Are fully compilable with instruction conversion & microcoded objects.

Allowing the system HPC to convert all available computation unity into realestate for high performance computing and gaming, rendering, Dynamic compiled code objects .. All at the same time.

Since compilers such as GCC shall have object compilers for Open CL Direct Compute code..
Direct from C++ & Fortrans & the objective code will be optimized for compute unite by class,

Float, Integer..

****

Data Example Chart: OpenCL Backed data object hive: data science, web pages and gaming


With pages in php with database backing rendered Though:
OpenCL Direct Compute maths scripting the output into Vulkan,ESGL & DirectX..

Into: object orientated data & charts, Into 2D/3D web pages & charts or diagrams.
Alternatively the rendering of High performance computing can compile and output:

Machine intelligence data, Medical data sets, Bioinformatics & gaming data informatics such as consoles & 3D Renderings.

Into: Web pages; Including JS,PHP & Database that back the dataset hive we need to improve the scientific readability & look.

****

However since float and integer exist together; Coding has become a slippery squiril so to speak..
Quite often floats in Vector code end up being translated from and to integer over and over,
The transfer of code from and to float is quite in-efficient,

However for example floating point definitions of page layout in PHP can lead to errors in web page layout & cost additional ram & page .php file size (however we do use compressed UTF-8 (GZIP),

Only the final form needs to definitely fit into the finite data set of integer or float ..
For rendering in high definition & VR realities we really need the 64bit precision or even float,
Data saved in integers saves money & resources such as storage..

Conversion of float data into integer & Integer into float takes advantage of resource allocations & we do need to ensure that output pipeline is float on higher than HD displays!

We can allocate OpenCL compatible code to integer units quite easily, But as stated we need clear lines between sets of integer code and float for optimisation reasons.

****
https://science.n-helix.com/2018/01/integer-floats-with-remainder-theory.html

https://science.n-helix.com/2016/04/3d-desktop-virtualization.html

https://science.n-helix.com/2017/02/open-gaming.html

Compiler books & reading : https://science.n-helix.com/2017/04/boinc.html

https://www.khronos.org/sycl/

https://llvm.org/

https://gcc.gnu.org/

(c)Rupert Summerskill

Thursday, May 16, 2019

Zombie Load bug update solution

15/05/2019 - Zombie load bug : Intel : said to slow down processors especially with java

https://www.pcgamesn.com/intel/zombieload-mds-vulnerability-security-patch-hyperthreading-mitigation-performance

How do we avoid the performance loss? Believable solve

Essentially we have to make speculative load cache private to the operating system at a minimum, essentially we can still use masked data load above the system but we need to verify the task ID and PID and where possible tab/Window or process ID.

Essentially we need to trim the dataset to the process in a tree ML.

Processor : Privileged execution by kernel : By application list & Privilege level in regard to the recurved data.

Memory Containment is not just prefetch stack but also system, OS & Process.

Mitigation by security dam , Masking data & Antivirus software.

Essentially prefetch data is necessary  for assemblers & coders to optimise the code stack,

However security privilege levels for accessing the code within the entire windows stack is to be prioritised by privacy level.

Programs that optimise the execution priority need access ideally to data on execution timeline & data fetch,

However accessing the applications memory array in random address space needs to be tailored to the type of execution; Who it is by,

Privilege level & the process that created the interception relative to the executed process.

While this may prove measurable in protection; low level kernel executed viruses would still be able to access above..

Masking in the form of up & down privilege priority and task child/father/mother is a complex machine learning theatre of war,

A field of operation requiring kernel & userland advanced & sophisticated cyber security,
Contributing elements such as memory encryption & key data field scrambling/masking from spying; Snooping and virus do also enable payloads to go unfound..

In short solutions that enable privacy for process are also to be enabled for antivirus & security threat detection.

Complex systems of personal protection will also have to scan for code; JS & other applicable code that is out of place within the appliance frame work / stack .. Without compromising the security/privacy we personally seek.

Masking data is a processing task subject to objective fair use policy & usable system operation; Optimisation ability; memory clearing; field reduction or use & re use; Personal & impersonal information or data subsets,

Tasks & management.

(c)Rupert S

Fix-Spectra.bat Enables patches in windows

https://science.n-helix.com/2018/01/microprocessor-bug-meltdown.html

Update 14/05/2019 or later installed and all VM's need to be shutdown and restarted & updated according to Microsoft post.

More details:

https://www.datacenterknowledge.com/security/here-s-how-zombieload-affects-data-centers-and-what-do-about-it

https://software.intel.com/security-software-guidance/insights/deep-dive-intel-analysis-microarchitectural-data-sampling

Update 2: Buffer security strategy


To obtain buffers for one application only..
Extra buffers are deployed, These buffers can be cleaned or contain application specific data.
These are program specific and contain only data for one program.

Remember that clear buffer fetching can be done from a single place involving a single cached fetch cycle and memory location modification on write / Memory Reloc & are to be in level 2 or 3 cache.

Thusally we are able to maintain a clear buffer, After all clear buffers are not program specific so one will do and hence a single fetch by cache.

As stated buffer security plans include localised buffer fetch sets, Application specific & secure.

Strategy 2

Buffer arrangement is tiered in strategy 2

Tier 1 : Tier 2 : Tier 3 : Tier 4 : Tier 5

The same way we draw an ML diagram <:> Core cache : Secure tier 1 : Secure Tier 2

The arrangement can be by PID & father daughter sets & does not necessitate the clearing of the buffer unless this is requires, In the case of a clean buffer a clear standard buffer is already in Cache & is swapped in.

This strategy avoids buffer clearing cycles directly interfering with the program execution cycle,

For buffers are ether program specific in a key ring or already clear / State flushed.

(C)RS

Friday, December 28, 2018

RNG Sources Examined

Entropy Ring Sources Examined


Chaos Entropy Pollinate Linux
Ubuntu cloud pollinators

"As can be seen there are still grand hopes of perfection, Sources can be purer..
Developments can surely be made to further improve the pool potential for RING Random/Chaos/Entropy sources
Further improvements to improve the future of cryptography are encouraged..
The signs are reliable progress will improve the system futures.

RS"

FIPS 140-2 successes: 1000 : 100% pure Chaos Rand : Ubuntu : perfectly random security : Updated

https://developers.cloudflare.com/argo-tunnel/reference/how-it-works/

/etc/cloudflared/config.yml

hostname: pollen-ub.n-helix.com
url: http://localhost
logfile: /var/log/cloudflared.log
compression-quality: 3
proxy-dns: true
proxy-dns-upstream:
 - https://1.1.1.1/dns-query
 - https://1.0.0.1/dns-query
proxy-dns-port: 53
autoupdate-freq: 6h
loglevel: debug
transport-loglevel: debug

https://github.com/dustinkirkland/pollinate

https://is.gd/Entropy

((100 bytes * 60) * 60) * 24 = 8.64 megabytes Rand per device ( Average is higher or lower than 100 )

21.3K+ Rand Entropy pollinate requests served in 12 hours average : #Pollinate #Ubuntu

AMD Ryzen & thread ripper have rand RNG/TPM units and are great pollinator nodes.
submit your pollinator nodes to security(at)ubuntu.com
use https://www.dynu.com/en-US/ to make a entropy.dyndns.net (for example) and send that to ubuntu security please.


Cryptographic Hardware RAND sources

"Why not Cloud providers like IBM, Amazon & google for RAND, Entropy .. Cryptographic TPM Hardware RND with great PRNG to improve security?
Cloud Providers are a most excellent source for Chaotic Entropy & provide true random chaotic sources for cryptographic ready RAND,
Additionally they provide most excellent resources to further the expansion of internet intellectual property.

By providing additional sources to the support at https://ubuntu.com the merits of expanding the data pool for linux research rand expound upon the needs of the many,
Requiring few manual regulatory examinations to certify the unlikely events leading to system insecurities or predictability,
The systems will be majorly improved on the factors of reliability & community drive.

By Creating the system that secures the internet Linux Ubuntu creates a key new market in the world of cyber security, Further forwards the science of Matter; Physics and Gaming,
Nexgen code has no barriers for greater utilisation of surprise & the fact we no longer have a clue what gaming plots really contain or what dynamic system flow will show about the efficiency and function of our technology..

Chaotic entropy enables the mapping of weather; Engine gas & other viscus fluid, liquid, gas and materials..
Secure internet & Enhanced cyclic butterfly effect creations, Research, Gaming.. Design & utility.

With this Chaotic Entropy & life expanding rand we will be able to do so much..
Thanks to Debian sourced linux distributions like Ubuntu we are able to produce..
100% certified chaos for research,Science,Engineering, Security & Gaming / Lotto / Casino Royal events..
Totally life changing.

RS"

*sources*

https://pollinate.n-helix.com/
https://pollinate2.n-helix.com/
https://entropy.n-helix.com/
https://entropy2.n-helix.com/

*science*

https://science.n-helix.com/2017/04/rng-and-random-web.html

****Statistics****

Delivery speed RND Pollinate : #Security
https://drive.google.com/open?id=178nvBqjBc_YOeKvavL32TNO_iNdlqNlN

AT Defence Plans
https://photos.app.goo.gl/reg3tkyhEV32P3NL9
https://drive.google.com/open?id=1iHSiZiRKgqxZF2akFFkvCthn1vTiYrsL

**** Statistics****

https://drive.google.com/open?id=1KKH_ejyWrjUnhZ0qp7swCjOaqGJs_FnR
https://drive.google.com/open?id=1O-HGjwxEZsQKwX_XWUKcbDj78zP-1r3Q
https://drive.google.com/open?id=1KKFjVyh73sn6IE9r5R6CH-iZp39R8LHU
https://drive.google.com/open?id=1TLPbbbcqaDTt2Ly1yb_uUXj6pK6XN4dJ
https://drive.google.com/open?id=1gOey570yjGBMcumCHCPTnt5iCeOCh7Y8
https://drive.google.com/open?id=1ZhVrsNdKmhlZ-dvqtnqyY5FuaQ9jLmxH
https://drive.google.com/open?id=1aD0OGlj_-lAlcGOcn3Z9jDPRo9kVHxFy
https://drive.google.com/open?id=1dGuG28cINIGhNHnvVkBLMmYMdmMgxDNp
https://drive.google.com/open?id=1284DyzPsQg1cac6KzFQvqJswPlFHNnpR
https://drive.google.com/open?id=1MGUavUApGQ6V6yDWEeI013hBnVShVetq
https://drive.google.com/open?id=1oFTqZPmdhtQmWw1NjigENrP5iCOW6lrs
https://drive.google.com/open?id=1w-HuYEN1gqvGWfkXQhDKu3JUKKnkLYrP
https://drive.google.com/open?id=1QFdb019lQ6PeFLAgNwv-VBVJoPOg7CyB
https://drive.google.com/open?id=1MQwLIC03ojKlMhxiqvS-Dm3pz-LHaiiV
https://drive.google.com/open?id=1cJkzc3_LQcLefopy7yyiLlLOH46bb-j9
https://drive.google.com/open?id=1j2KhSMGg0Pza77d0QDRn-XwV846Ob5ft
https://drive.google.com/open?id=1s6p5kE9ZZHduaNgjoVa5GpA4WFJXfbJH
https://drive.google.com/open?id=1y-gkCY0ZwKt-rp82ReSUAFhnlQJWyuCV

FIPS Records 2019-01-07

https://drive.google.com/open?id=1E8gxUN0NfmDGNT-eDK46sREySUrBlP67

**** Statistics****

sonex-reg@n-helix-com:~$ cat /proc/sys/kernel/random/entropy_avail
3555
sonex-reg@n-helix-com:~$ cat /dev/random | rngtest -c 40000
rngtest 5
Copyright (c) 2004 by Henrique de Moraes Holschuh

rngtest: starting FIPS tests...
rngtest: bits received from input: 800000032
rngtest: FIPS 140-2 successes: 39965
rngtest: FIPS 140-2 failures: 35
rngtest: FIPS 140-2(2001-10-10) Monobit: 4
rngtest: FIPS 140-2(2001-10-10) Poker: 1
rngtest: FIPS 140-2(2001-10-10) Runs: 11
rngtest: FIPS 140-2(2001-10-10) Long run: 19
rngtest: FIPS 140-2(2001-10-10) Continuous run: 0
rngtest: input channel speed: (min=959.531; avg=14970.406; max=19531250.000)Kibits/s
rngtest: FIPS tests speed: (min=1.630; avg=82.952; max=161.640)Mibits/s
rngtest: Program run time: 61397903 microseconds
sonex-reg@n-helix-com:~$ cat /proc/sys/kernel/random/entropy_avail
3193
sonex-reg@n-helix-com:~$ top -c

top - 21:00:44 up 21:01,  1 user,  load average: 3,07, 3,23, 5,04
Tasks: 325 total,   1 running, 245 sleeping,   0 stopped,   1 zombie
%Cpu(s): 10,8 us,  1,5 sy,  0,0 ni, 87,4 id,  0,1 wa,  0,0 hi,  0,2 si,  0,0 st
KiB Mem : 24691920 total, 12718944 free,  8498344 used,  3474632 buff/cache
KiB Swap: 41921548 total, 41921548 free,        0 used. 15723984 avail Mem 

A cascade of mono-bit shredders ... and long runs show considerable thought is required for RND development.

**

sonex-reg@n-helix-com:~$ cat /proc/sys/kernel/random/entropy_avail
3545
sonex-reg@n-helix-com:~$ cat /dev/random | rngtest -c 25000
rngtest 5
Copyright (c) 2004 by Henrique de Moraes Holschuh

rngtest: starting FIPS tests...
rngtest: bits received from input: 500000032
rngtest: FIPS 140-2 successes: 24981
rngtest: FIPS 140-2 failures: 19
rngtest: FIPS 140-2(2001-10-10) Monobit: 2
rngtest: FIPS 140-2(2001-10-10) Poker: 3
rngtest: FIPS 140-2(2001-10-10) Runs: 5
rngtest: FIPS 140-2(2001-10-10) Long run: 9
rngtest: FIPS 140-2(2001-10-10) Continuous run: 0
rngtest: input channel speed: (min=1.117; avg=14.647; max=19073.486)Mibits/s
rngtest: FIPS tests speed: (min=3.604; avg=81.979; max=161.640)Mibits/s
rngtest: Program run time: 38381555 microseconds
sonex-reg@n-helix-com:~$ cat /proc/sys/kernel/random/entropy_avail
3146
sonex-reg@n-helix-com:~$ top -c

top - 09:54:44 up 13:35,  1 user,  load average: 3,79, 3,78, 3,21
Tasks: 359 total,   1 running, 280 sleeping,   0 stopped,   1 zombie
%Cpu(s):  8,9 us,  2,4 sy,  0,0 ni, 83,0 id,  5,6 wa,  0,0 hi,  0,1 si,  0,0 st
KiB Mem : 24691920 total,  9494520 free, 10551780 used,  4645620 buff/cache
KiB Swap: 41921548 total, 41921548 free,        0 used. 13641316 avail Mem

sonex-reg@n-helix-com:~$ cat /proc/sys/kernel/random/entropy_avail

3477
sonex-reg@n-helix-com:~$ cat /dev/random | rngtest -c 40000
rngtest 5
Copyright (c) 2004 by Henrique de Moraes Holschuh

rngtest: starting FIPS tests...
rngtest: bits received from input: 800000032
rngtest: FIPS 140-2 successes: 39975
rngtest: FIPS 140-2 failures: 25
rngtest: FIPS 140-2(2001-10-10) Monobit: 0
rngtest: FIPS 140-2(2001-10-10) Poker: 7
rngtest: FIPS 140-2(2001-10-10) Runs: 11
rngtest: FIPS 140-2(2001-10-10) Long run: 7
rngtest: FIPS 140-2(2001-10-10) Continuous run: 0
rngtest: input channel speed: (min=1.075; avg=14.661; max=19073.486)Mibits/s
rngtest: FIPS tests speed: (min=5.699; avg=78.753; max=160.281)Mibits/s
rngtest: Program run time: 61738871 microseconds
sonex-reg@n-helix-com:~$ cat /proc/sys/kernel/random/entropy_avail
3211
sonex-reg@n-helix-com:~$ top -c

top - 17:08:56 up 1 day, 16:42,  1 user,  load average: 5,08, 5,28, 4,39
Tasks: 383 total,   1 running, 304 sleeping,   0 stopped,   2 zombie
%Cpu(s): 10,4 us,  2,4 sy,  0,0 ni, 81,2 id,  5,9 wa,  0,0 hi,  0,1 si,  0,0 st
KiB Mem : 24691920 total,   939216 free, 19678264 used,  4074440 buff/cache
KiB Swap: 41921548 total, 41921548 free,        0 used.  4331756 avail Mem


sonex-reg@n-helix-com:~$ cat /proc/sys/kernel/random/entropy_avail
3519
sonex-reg@n-helix-com:~$ cat /dev/random | rngtest -c 1000
rngtest 5
Copyright (c) 2004 by Henrique de Moraes Holschuh

rngtest: starting FIPS tests...
rngtest: bits received from input: 20000032
rngtest: FIPS 140-2 successes: 1000
rngtest: FIPS 140-2 failures: 0
rngtest: FIPS 140-2(2001-10-10) Monobit: 0
rngtest: FIPS 140-2(2001-10-10) Poker: 0
rngtest: FIPS 140-2(2001-10-10) Runs: 0
rngtest: FIPS 140-2(2001-10-10) Long run: 0
rngtest: FIPS 140-2(2001-10-10) Continuous run: 0
rngtest: input channel speed: (min=990.378; avg=15237.623; max=19531250.000)Kibits/s
rngtest: FIPS tests speed: (min=40.930; avg=75.822; max=145.599)Mibits/s
rngtest: Program run time: 1533851 microseconds
sonex-reg@n-helix-com:~$ cat /proc/sys/kernel/random/entropy_avail
3157
sonex-reg@n-helix-com:~$ top -c

top - 20:03:59 up 19:37,  1 user,  load average: 1,53, 2,33, 2,57
Tasks: 378 total,   2 running, 294 sleeping,   0 stopped,   2 zombie
%Cpu(s): 10,6 us,  2,5 sy,  0,0 ni, 81,3 id,  5,5 wa,  0,0 hi,  0,1 si,  0,0 st
KiB Mem : 24691920 total,  6488632 free, 14951916 used,  3251372 buff/cache
KiB Swap: 41921548 total, 41921548 free,        0 used.  9183856 avail Mem

****
sonex-reg@n-helix-com:~$ cat /proc/sys/kernel/random/entropy_avail
3617
sonex-reg@n-helix-com:~$ cat /dev/random | rngtest -c 1000
rngtest 5
Copyright (c) 2004 by Henrique de Moraes Holschuh

rngtest: starting FIPS tests...
rngtest: bits received from input: 20000032
rngtest: FIPS 140-2 successes: 1000
rngtest: FIPS 140-2 failures: 0
rngtest: FIPS 140-2(2001-10-10) Monobit: 0
rngtest: FIPS 140-2(2001-10-10) Poker: 0
rngtest: FIPS 140-2(2001-10-10) Runs: 0
rngtest: FIPS 140-2(2001-10-10) Long run: 0
rngtest: FIPS 140-2(2001-10-10) Continuous run: 0
rngtest: input channel speed: (min=303.861; avg=15125.460; max=9765625.000)Kibits/s
rngtest: FIPS tests speed: (min=24.328; avg=63.635; max=145.599)Mibits/s
rngtest: Program run time: 1591605 microseconds
sonex-reg@n-helix-com:~$ top -c

top - 14:12:57 up 1 day, 13:59,  1 user,  load average: 3,32, 3,57, 3,56
Tasks: 415 total,   3 running, 332 sleeping,   0 stopped,   1 zombie
%Cpu(s): 14,2 us,  3,4 sy,  0,0 ni, 81,5 id,  0,7 wa,  0,0 hi,  0,2 si,  0,0 st
KiB Mem : 16434388 total,   355552 free, 13389264 used,  2689572 buff/cache
KiB Swap: 41921548 total, 40637708 free,  1283840 used.  1393852 avail Mem

Rupert S

Friday, September 21, 2018

HPC Pack install guide

HPC Pack install guide : Science,Server & Web : Rupert S

https://www.microsoft.com/en-us/download/details.aspx?id=54507 Microsoft HPC Pack 2016 including linux

Router/Firewall ports: 53,80,443,1812,1813,2408,3128,3130,5353,7844,8080,11211

Router/Firewall ports HPC Pack: 53,80,443,1812,1813,1856,2408,3128,3130,5353,5800,5801,5969,5970,5974,5999,6729,6730,7844,7997,8677,8080,9087,9090,9091,9092,9094,9095,9096,9794,9892,9893,11211


https://is.gd/HPCData_P

Windows server: Step 1


install Microsoft HPC Pack - Microsoft HPC Pack server/Workstation/Unmanaged server/node

Head Node
127.0.0.1,dn.n-helix.com,worldcommunitygrid.org,lhcathome.cern.ch,cern.ch,boinc.bakerlab.org,cloudflareworkers.com,cloudflare.com,cloud.sfr.fr,sfrcloud.sfr.fr,webrlb01.cern.ch,boincai01.cern.ch,8.8.4.4,1.1.1.1


You will need a server certificate such as a comodo or digicert or microsoft.. for the next page of hpc pack install

https://www.gogetssl.com/compare-domain-sslcertificates/
https://certifytheweb.com/home/
https://letsencrypt.org/docs/client-options/

Docker though the install service

powershell Install-PackageProvider -Name NuGet -Force
powershell Install-Module -Name DockerMsftProvider -Repository PSGallery -Force
powershell Install-Package -Name docker -ProviderName DockerMsftProvider -Force

or run one of these instead of step 1 & 2:
For Intel GPU : https://is.gd/DockerIntelServerInstall
For Nvidia GPU : https://is.gd/DockerNvidiaServerInstall
For AMD GPU : https://is.gd/DockerAMDServerInstall

Windows 7/8/10: Step 1


install Microsoft HPC Pack node or workstation/server

Head Node
127.0.0.1,dn.n-helix.com,worldcommunitygrid.org,lhcathome.cern.ch,cern.ch,boinc.bakerlab.org,cloudflareworkers.com,cloudflare.com,cloud.sfr.fr,sfrcloud.sfr.fr,webrlb01.cern.ch,boincai01.cern.ch,8.8.4.4,1.1.1.1


You will need a server certificate such as a comodo or digicert or microsoft.. for the next page of hpc pack install

https://www.gogetssl.com/compare-domain-sslcertificates/
https://certifytheweb.com/home/
https://letsencrypt.org/docs/client-options/

https://docker.com - docker CE for windows 7/8/10 install requiring download and install of docker community edition edge..

powershell Install-PackageProvider -Name NuGet -Force
powershell Install-Module -Name DockerMsftProvider -Repository PSGallery -Force
powershell Install-Package -Name docker -ProviderName DockerMsftProvider -Force
and then install docker CE

https://docker.com

run one of these after installing hpc pack & docker: Instead of Step 2

Step 2 for both : Enable Kubernets in docker and set available CPU's & Resources then >

or do the admin : CMD List or run the script above >
admin : CMD

docker run --name railgun-memcached -d -p=127.0.0.1:11211:11211 -e https_proxy=http://127.0.0.1:3128 -e http_proxy=http://127.0.0.1:3128 -e ftp_proxy=http://127.0.0.1:3128 --restart=always memcached:latest

docker run -d --name=railgun -p 2408:2408 -e RG_ACT_TOKEN=ENTERapiTOKENHERE -e RG_ACT_HOST=127.0.0.1 -e RG_LOG_LEVEL=1 -e RG_WAN_PORT=2408 -e https_proxy=http://127.0.0.1:3128 -e http_proxy=http://127.0.0.1:3128 -e ftp_proxy=http://127.0.0.1:3128 --link railgun-memcached:memcached --restart=always rungeict/cloudflare-railgun

docker run -d --name=squid-memcached -p=127.0.0.1:3128:3128 -e RG_ACT_HOST=127.0.0.1 -e RG_LOG_LEVEL=1 -e https_proxy=http://127.0.0.1:3128 -e http_proxy=http://127.0.0.1:3128 -e ftp_proxy=http://127.0.0.1:3128 --link railgun-memcached:memcached --restart=always  minimum2scp/squid

docker run -d --name=nuster-memcache -p=127.0.0.1:8080:8080 -p=127.0.0.1:443:443 -p=127.0.0.1:80:80 -e RG_ACT_HOST=127.0.0.1 -e RG_LOG_LEVEL=1 -e https_proxy=http://127.0.0.1:3128 -e http_proxy=http://127.0.0.1:3128 -e ftp_proxy=http://127.0.0.1:3128 --restart=always nuster/nuster

docker run --name binddns-resolver -d --restart=always -e https_proxy=http://127.0.0.1:3128 -e http_proxy=http://127.0.0.1:3128 -e ftp_proxy=http://127.0.0.1:3128 -p 53:53/tcp -p 53:53/udp -p 5353:5353/tcp -p 5353:5353/udp -p 10000:10000/tcp --volume /srv/docker/bind:/data sameersbn/bind

# to configure and update binddns-resolver : https://localhost:10000

# alternative requires to be configured docker run --name=binddns-resolver -d -p 53:53/udp -p 53:53 -d -p 5353:5353/udp -p 5353:5353 -e https_proxy=http://127.0.0.1:3128 -e http_proxy=http://127.0.0.1:3128 -e ftp_proxy=http://127.0.0.1:3128 -d --restart=always --volume /srv/docker/bind:/data unixtastic/bind-dnssec-resolver

# Do not forget that the first DNS server is 127.0.0.1

docker run --name nax-waf -e BACKEND_IP=127.0.0.1 -e KIBANA_PASSWORD=Safe5 -d --link railgun-memcached:memcached --restart=always scollazo/naxsi-waf-with-ui

docker exec squid-memcached curl http://127.0.0.1/

****

When you enable Kubernet in docker in the console (CMD) : kubectl: Kubernets ingress controller security: If you use cloudflared & or a local certificate : the HPC Certificate you got: https://kubernetes.io/


(CMD):

kubectl create secret generic 127.0.0.1 --from-file="$HOME/.cloudflared/cert.pem"

kubectl create secret generic localhost --from-file="$HOME/.cloudflared/cert.pem"

****



****

Selenium runs once - you need to re run it on boot

https://store.docker.com/community/images/selenium/standalone-chrome
https://github.com/SeleniumHQ/docker-selenium

docker run -d -it --rm -p=0.0.0.0:9222:9222 -p=0.0.0.0:443:443  --link railgun-memcached:memcached --name=chrome-headless-selenium -v /tmp/chromedata/:/data selenium/standalone-chrome

port : 4444


****
*waf firewall*
https://hub.docker.com/r/scollazo/naxsi-waf-with-ui/
docker pull scollazo/naxsi-waf-with-ui

*zend php stack is licenced*
https://hub.docker.com/_/php-zendserver/
docker pull php-zendserver

*nginx php stack*
https://hub.docker.com/r/richarvey/nginx-php-fpm/
docker pull richarvey/nginx-php-fpm

*node js*
https://hub.docker.com/r/mhart/alpine-node/
docker pull mhart/alpine-node

*squid cache*
https://hub.docker.com/r/minimum2scp/squid/
docker pull minimum2scp/squid
http://www.squid-cache.org/Intro/


*Nuster http Cache - needs configuration !*
https://github.com/jiangwenyuan/nuster
https://hub.docker.com/r/nuster/nuster/
docker pull nuster/nuster

*railgun*
https://www.cloudflare.com/docs/railgun/installation.html
https://www.cloudflare.com/docs/railgun/daemon_execution.html

https://hub.docker.com/r/rungeict/cloudflare-railgun/
docker pull rungeict/cloudflare-railgun

https://www.cloudflare.com/website-optimization/railgun/?r=1

*bind DNS*
https://hub.docker.com/r/sameersbn/bind/#introduction
docker pull sameersbn/bind

# to configure and update binddns-resolver : https://localhost:10000


****
dhcp servers

freeradius dhcp servers
https://hub.docker.com/r/freeradius/freeradius-server/
docker pull freeradius/freeradius-server
docker run -d --name my-radius -p 1812-1813:1812-1813/udp --restart=always freeradius/freeradius-server

RackHD dhcp server
https://rackhd.readthedocs.io/en/latest/
https://github.com/RackHD/
https://hub.docker.com/r/rackhd/isc-dhcp-server/
docker pull rackhd/isc-dhcp-server

docker run -d --name rackhd-dhcp -p 1812-1813:1812-1813/udp --restart=always rackhd/isc-dhcp-server

Alpine dhcp
https://hub.docker.com/r/joebiellik/dhcpd/
docker pull joebiellik/dhcpd
****

https://hackernoon.com/a-kubernetes-guide-for-docker-swarm-users-c14c8aa266cc

https://blog.alexellis.io/you-need-to-know-kubernetes-and-swarm/

Further reading for swarm node configuration

https://github.com/docker/labs/blob/master/swarm-mode/beginner-tutorial/README.md

Traefik : Swarm cloud single ip/site url; header wall that keeps all nodes hidden within to handle web traffic from the singular web portal to inside the node array list (funnels Data, Hence > traefik)
https://github.com/bee42/traefik-with-docker
https://github.com/bee42/docker-orchestration-swarming

https://hub.docker.com/r/bee42/docker-orchestration-swarming/
docker pull bee42/docker-orchestration-swarming

****
Open science grid including tensor flow & GPU
https://hub.docker.com/u/opensciencegrid/

swarm elasticsearch
https://hub.docker.com/r/agoryachev/docker-swarm-elasticsearch/

****
Media server
https://hub.docker.com/r/kurento/kurento-media-server/
docker pull kurento/kurento-media-server
docker run -d --name kms-mediaserver -p 8888:8888 kurento/kurento-media-server

Image neural enhancer
https://github.com/alexjc/neural-enhance
https://hub.docker.com/r/alexjc/neural-enhance/
docker pull alexjc/neural-enhance

caffe gpu
https://hub.docker.com/r/tleyden5iwx/caffe-gpu-master/
docker pull tleyden5iwx/caffe-gpu-master

****
Boinc HPC Computing

https://github.com/BOINC/boinc-client-docker

https://github.com/marius311/boinc-server-docker/blob/master/docs/cookbook.md

https://cosmicmar.com/2016/02/14/boinc-server-docker-1/

Our suggestion of boinc detection of docker (docker launch takes 15 seconds plus so boinc would have to wait for the process to initiate in the windows task manager before..

The initial detection system would launch for Container VM & also pull user data into the VM (the necessary ID from project add device & also set the VM and OS use for special case use and non generic project data,
Of special note is the use of Linux containers; This container potentially could batch multiple projects with finesse; In particular Science Linux and Alpine (for size and usability) (science kernel version is possible)

docker pull alpine - check this out!

*alpine is a tiny Linux with many apps very flexible 5mb*
https://hub.docker.com/_/alpine/
docker pull alpine


Finally special contained secure environments for HPC Boinc Networking science is realised with relative technical aplume & thusly easily usable by the average; Non average user.

boinc node network on alpine linux with rasberry-pi
https://github.com/firepear/homefarm

****
http://science.n-helix.com/2018/09/hpc.html - a bit about boinc in terms of Docker & Microsoft HPC Pack
http://science.n-helix.com/2017/04/boinc.html - HPC, Optimisation & Boinc - programming

****
Squid for Windows & Linux

http://www.squid-cache.org/

easy install for windows
http://squid.diladele.com/
http://packages.diladele.com/squid/3.5.28/squid.msi

https://techencyclopedia.wordpress.com/2018/04/07/how-to-install-and-configure-squid-proxy-on-windows/

Squid.conf for easy setup of squid settings for cache > https://is.gd/Squid_conf

****
Some ambitious specs for a VM

8 cores, 8Gb ram, 1 GPU (2gb ram), 20GB HDD per process ^^

That being said : Hyper V VM allows Dynamic ram as an option & this means the ram can be used by the system if the VM is not using it..

Docker has a similar method in that the Docker VM uses a pre-defined space :
In my case 16gb and 8 cores, This is sub allocated along with linux swap ..
To processes in containers, Containers can use most varieties of linux including alpine (very small memory footprint) Alpine is suitable for science projects as is the Docker Kernel.

RS