Super micro AI infrastructure solutions aim to transform CSPs

With more than 5,000 providers worldwide, cloud service providers face a highly competitive landscape and must ensure their AI infrastructure solutions align with market demands. These CSPs must stand out with superior performance, cost-effective solutions and innovative offerings to succeed.

With that in mind, Super Micro Computer Inc. and its partners have focused on developing advanced technologies to meet the evolving demands of AI and high-performance computing. Supermicro is focused on delivering high-performance, energy-efficient solutions in partnership with Vast Data Inc., Advanced Micro Devices Inc. and Solidigm, according to Ben Lee (pictured, front row, left), director of solution architect and business development at Super Micro.

Ben Lee of Super Micro Computer; Sachin Hindupur of AMD; Neeloy Bhattacharyya of Vast Data; and Bill Panos of Solidigm talk with theCUBE’s Rob Strechay.

“These collaborations enable [cloud service providers] to meet the demanding storage you need for AI workloads,” he said.

Lee; Sachin Hindupur (back row, right), global strategy leader at AMD; Neeloy Bhattacharyya (front row, right), director of AI/HPC solutions engineering at Vast; and Bill Panos (back row, left), senior product marketing engineering manager at Solidigm, spoke with theCUBE Research’s Rob Strechay at the Supermicro Open Storage Summit series, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio.  They discussed the challenges and opportunities for cloud service providers in managing AI and high-performance computing workloads, focusing on the role of AI infrastructure solutions and cutting-edge technologies in meeting these demands. (* Disclosure below.)

A goal to assist cloud providers with AI infrastructure solutions

Today, cloud service providers are trying to figure out how their traditional infrastructure has been built out. For Vast, the experience with CSPs has revealed a focus on relevance, brand and cost, according to Bhattacharyya.

“When it comes to relevance, a lot of the workload, especially the AI workload that the CSPs have been encountering to date, have been centered around very large GPU infrastructures and model training,” he said. “That’s been the primary workload. While there’s an entire group of boutique AI-focused CSPs that have emerged from that, at the end of the day, the margin profiles that come with bare metal GPU as a service are just, candidly, not that great.”

CSP customers have been looking at how to be responsive to model trainers and enterprise customers looking to start to actually deploy and serve from the models. That’s where the Vast Data Platform comes into play, according to Bhattacharyya, “both on the data preparation side, with some of our ability to accelerate workloads like Spark, and then on the inference side, which needs a lot of distributed capacity and access to models and vector databases at many different locations to modulate the latency of the workload.”

Uptime is key to brand and reputation. That’s why Vast has sought to provide an always-on solution, Bhattacharyya added.

“You never have to arrange for any downtime,” he said. “In addition, especially when it comes to enterprise customers who are conscious about security, who are conscious about compliance auditing and traceability of what data went into what model and what was used to generate what result, it’s another area where we’ve been partnering really closely with the CSPs to ensure that they can provide a really leading-edge offering to their customers.”

Prioritizing cost efficiency and reliable AI infrastructure solutions

Meanwhile, cloud service providers are always looking to increase their margins while lowering the operating costs. Downtime can be costly for data centers and cloud service providers, making reliable support crucial, according to Hindupur.

“You can have, for example, engineering support, you can have field application engineers, you can have also customer-related support,” he said. “More importantly, you have support for the products, depending on the product which is there. It can be all the way from five to 10 years. So [AMD is] very well suited … to meet the needs of the storage customers.”

In helping CSPs get the most from their AI-based solutions, there are a number of things to keep in mind. Among those involves the need for storage, as well as the need for high density, according to Panos, “especially high density per watt. And that drives this performance model and this TCO model that we’re wanting to deliver to CSPs, so they can deliver that to their customers.”

While Solidigm today offers a 61.44 terabyte product, the company is looking to go further in the future. But parts will remain consistent, Panos added.

“We want to keep it in relatively the same form factor as much as we can to have that extremely high density,” he said. “There’s a lot of different things that we’re doing to optimize to the best of our ability.”

Stay tuned for the complete video interview, part of SiliconANGLE’s and theCUBE Research’s coverage of the Supermicro Open Storage Summit.

Photo: SiliconANGLE

(* Disclosure: TheCUBE is a paid media partner for the Supermicro Open Storage Summit event. Neither Super Micro Computer Inc., the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)