This article will look at a scalable orchestrated high-performance computing approach, including storage, as well as Qumulo and WDC storage used in public health information.  We also look at a VAST and NVIDIA high performance AI workloads as well as Hitachi Vantara and Cloudian Kubernetes announcements.

Berkeley, CA start-up Ctrl IQ announced a way to merge the key capabilities of enterprise, hyper-scale, cloud and high-performance computing (HPC).  The company says that its multi-cloud, multi-site and performance-focused orchestration solution opens access to artificial intelligence, machine learning, analytics and scientific workflows everywhere.

The company was founded by Gregory Kurtzer, who has done extensive work in the HPC industry and was the creator of open source projects CentOS and Rocky Linux.  The company recently closed on a $4M Series A funding round from storage company, OpenDrives, and IAG Capital Partners.  IAG Capital Partners was recently announced as the lead investor in a $20M funding round for OpenDrives.

The COVID-19 pandemic and AI research has driven HPC demand to new heights.  For instance, modelling the COVID virus’ spike protein and finding sites on those spikes that could be drug targets are done with HPC-powered simulation.

The figure below shows Ctrl IQ’s clustered HPC structure compared to a traditional HPC flat architecture.  In the Ctrl IQ approach common open source and containerized workflows are supported that access data and compute resources from on-premises, the cloud and even edge devices.  Note the inclusion of OpenDrives storage.  Using this approach for orchestrated rack-based units allows hyper-scale data center HPC architectures that support processing and storage to solve big data problems.

Ctrl IQ’s fully supported technology stack was created to accelerate the speed and efficiency of multi-premises, multi-cloud and multi-architecture orchestration of workflows. The solution fits within existing enterprise architecture and infrastructures (e.g. in standard data center racks), and will include:

·       Rocky Linux: enterprise Linux supported for cloud and enterprise

·       Warewulf: scalable systems management suite designed to manage clusters of compute resources.

·       Ctrl Computing Stacks: fully supported computational stacks designed for all performance intensive workflows like, HPC, AI, ML and scientific workflows.

·       Fuzzball: provides the intelligent, secure and performant orchestration of workflows and data in a managed or hosted on-premise solution.

·       Ctrl IQ Cloud: provides a secure hybrid platform to execute and orchestrate both performance critical workflows and services.

Speaking of using storage to solve big data problems, Western Digital Corp. announced that the Institute for Health Metrics and Evaluation (IHME) based at the University of Washington’s School of Medicine, in partnership with Qumulo’s file data management for hybrid cloud environments, is employing WDC’s Ultrastar DC HC550 18TB HDDs and Ultrastar DC SN640 NVMe SSD for ongoing COVID-19 research and vaccine rollout.  IHME provides public health information to policymakers and healthcare institutions to help them make informed public health decisions.

Qumulo’s C-432T offers a cached-performance hybrid storage system for data center, private and public cloud environments. This high-performance system yields a dynamically scalable architecture and real-time visibility of data.   With Western Digital’s vertically integrated storage devices, Qumulo’s C-432T provides 432TB of raw storage capacity in a 2U configuration.

VAST announced a joint reference architecture based upon NVIDIA’s DGX A100 systems and VAST Data’s Universal Storage platform.  This reference architecture is designed to significantly increase storage performance for AI use cases such as large-scale training of conversational AI models and petabyte-scale data analytics.  Built on VAST’s LightSpeed platform, the architecture leverages VAST’s NFS-over-RDMA, NFS multipath and NVIDIA’s GPUDirect Storage as well as a converged fabric design.  The joint reference architecture delivers over 140 GB/s throughput for both GPU-intensive and storage-intensive AI workloads.

Hitachi Vantara announced the availability of its Hitachi Kubernetes Service to tackle the challenge of managing the complexity of multiple Kubernetes environments.  The company says that Hitachi Kubernetes Services enables customers to simply, consistently, and securely deploy, manage, monitor, and govern Kubernetes clusters across major cloud providers and on premises simply, consistently, and securely. This empowers developers to deploy workloads on their platform of choice, and eliminates vendor lock-in.

Cloudian announced support for container-based applications via a new Cloudian Kubernetes S3 Operator, providing a self-service tool for developers to dynamically provision Cloudian HyperStore S3-compatible object storage.  The company is also offering fully native S3-compatible object storage software for VMware Cloud Foundation with VMware Tanzu.  Cloudian also announced a solution based upon its HyperStore object storage and the Veeam Availability Suite v10 that provides ransomware protection within an end-to-end on premises backup architecture.

Ctrl IQ wants to make HPC mainstream.  Qumulo and WDC provide storage for public health.  VAST and NVIDIA have a 170GB/s joint reference architecture.  Hitachi Vantara provides a Kubernetes service.  Cloudian also announces Kubernetes support, compatibility with VMware cloud foundation and ransomware protection.

By lan