Google Cloud TPU: GCP Takes On AWS With The New AI Chips

With the invent of machine learning, the majority of companies are now exploring options on how to use the technology for their business benefits. From forecasting customer demands to projecting stock markets to identifying risks – majority of the use cases of machine learning revolves around crunching a mammoth data set and presenting the inferences for decision making at a lightning speed – not once but again and again and again.

Where the nature of the developer’s query is critical, the challenge also lies on the server to support the running of such a query involving such huge data sets.

This is where technologies like Google Cloud TPUs play a critical role. Cloud TPUs is a step forward in machine learning which would ease the enormous amount of computation, for researchers, engineers, and data scientists.

The Cloud TPU – formerly known as the TPU2 – is the second generation of Google’s homegrown math accelerators geared towards AI and machine learning workloads that rely on TensorFlow. While the first generation was only applicable for training neural networks, the Cloud TPU can handle both training and inference.

What is Google Cloud TPU?

According to Google, "Tensor Processing Units (TPUs) are Google’s custom-developed application-specific integrated circuits (ASICs) used to accelerate machine learning workloads."

Google’s Cloud Tensor Processing Unit (TPU) chips – formerly known as the TPU2 – is the second generation of Google’s homegrown math accelerators geared towards AI and machine learning workloads that rely on TensorFlow. While the first generation was only applicable for training neural networks, the Cloud TPU can handle both training and inference.

In layman terms, the TPUs are processing units specifically designed for real-time matrix multiplication in machine learning. It is a very fast processing unit which can process up to 180 Teraflops (a unit of computing speed equal to one million floating-point operations per second.) per TPU. It also saves power usage as it can perform 30-80x operations per watt.

As the TPU is known for its great speed and less power consumption with the ability to solve dense vector and matrix computations in time and with precision, it can mostly be used for the complex structure. It is specifically used for neural network workloads, very large models with very large effective batch sizes etc.

How does Google Cloud TPU help GCP?

Opening up Google’s can be a great way to consolidate Google Cloud Platform (GCP’s) position against AWS. Google had been fighting a brave battle to gain market share in the cloud hosting space and cloud TPUs abilities are a sure draw for some product makers.

Google Cloud’s ultimate objective of offering its customers options for their machine learning workloads that includes a variety of high-performance CPUs, including Skylake, and GPUs, like the Tesla V100 accelerator, along with the Cloud TPUs, seems like an attractive proposition.

Also, at a price that Google rolled this out - $6.50 USD / Cloud TPU / hour – may seem attractive to many. Closest benchmark will be with AWS P3.4Xlarge which comes at $12.24 /hour.

The AI-based Cloud competition has heated up but the war is far from over. We are yet to see how market leader AWS responds to the situation.