Cloud Service Providers Moving To Custom Hardware? Here’s Everything You Need To Know

Abhisheik Shandilya

5 months ago

Cloud Service Providers (CSPs) use generic hardware to benefit from economies of scale and keep things uniform in general. However, CSPs might be looking to change this trend. See, it is not just the consumer tech sector where hardware wars are being fought; even CSPs are trying to move toward custom hardware.

Why are Cloud Providers Moving To Custom Hardware?

Since the generic hardware manufacturers have been unable to keep up with the rapidly increasing demands of Cloud industry, CSPs are looking to enter the hardware space. The rise of highly scalable AI (Artificial Intelligence) and Machine Learning (ML) workloads, as well as IoT (Internet of Things) and analytics applications, is pushing CSPs toward new paths. The increased workload demands new architecture and generic makers have been unable to keep up with the pace.

The basic demand fueling this shift away from generic hardware makers is the need for innovative software models capable of supporting large data volumes. CSPs want to fulfill the need of their customers and at affordable costs. The evergreen demand for increased computing power at lower costs has exposed the ceiling of current hardware being used by CSPs.

One can safely bet that CSPs will make every effort to achieve better efficiency for their infrastructure. This efficiency will not only provide them with increased capabilities but will also reduce costs. The reduced costs, along with increased capabilities, can then be theoretically passed on to their customers. Put this in the picture with the ceiling that current generic hardware comes with, and it becomes clear why CSPs are trying to move to custom hardware.

Where Does the Industry Currently Stand?

While the move to custom hardware is continuously evolving and is currently in its initial stages, certain CSPs have already taken strides. AWS already owns an Israeli semiconductor maker, and Google operates its own Tensor Processing Units (TPUs). Just like the Cloud market overall, AWS has the lead in custom hardware too. AWS already has its Nitro Security Chip that not only increased efficiency in virtual machine provisioning overall but also made sure that AWS alone can update its firmware. Further, the Seattle-giant also brought out more efficient routers for its data centers.

Among the most recent moves of AWS has been developing AWS Inferentia, a customer inference engine for AI, and the ARM-based Graviton line of CPUs. As far as the current tide is concerned, CSPs are actively interested in third-part solutions for Cloud hardware architectures to fuel accelerated AI workloads and reduce costs, or even both. The future also looks bright as there are several AI chip and quantum computing startups with potential solutions that could see usage in the Cloud in the coming days. Further, these up and comers are also working on new chips to optimize Cloud infrastructure.

What Does Custom Hardware Hold for the Industry?

As already discussed, the move toward custom hardware is fueled with the potential of increased computing power and lower costs. Another aspect that the CSPs are focusing on via custom hardware is reduced latency. The current COVID-19 hit world has brought in increased work-from-home demands, along with a huge spike in services such as remote learning, gaming, and video-conferencing, among others. This has brought scalability challenges for CSPs as their infrastructure is under constant pressure to over-perform.

Another significant area that custom hardware promises to deliver on is making inroads into Bare Metal category. Bare Metal digital infrastructure allows secure and single-tenant hardware that can be geographically distributed for better performance. While the traditional method for setting up Bare Metal infrastructure involves significant expenses and commitment by businesses, Cloud-based Bare Metal eliminates these concerns. Lastly, custom hardware for Cloud also has the potential to redefine traditional computing architecture for AI, among others.

Overview

While pregnant with huge potential for CSPs, custom hardware is still in its early stages. Not to mention that custom hardware will also need developers and learning new languages to work on the hardware. Another aspect that needs to be addressed is the possible creation of a ‘walled-garden.’ CSPs might be tempted to use proprietary technology with their own hardware (if the move succeeds), and it could create walled-gardens for their customers as they’ll be effectively locked-in with the service. Further, this would also theoretically make migration to any other CSP a challenging task. Nonetheless, custom hardware for CSPs is a developing issue, watch this space for updates on this topic.