Google Cloud is introducing new AI chips to compete with Nvidia

Google Cloud Wednesday he announced that its eighth generation of AI chips, or tensor processing units (TPUs), will be split into two. One chip, called the TPU 8t, will be designed for model training and the other, the TPU 8i, will be for monitoring.

Inference is the continuous use of models, which occurs when users send instructions.

As you might expect, a the company is doing other interesting features of these new TPUs compared to previous generations: up to 3x faster AI model training, 80% efficiency per dollar, and the ability to get 1 million + TPUs to work together in one group. The ride should be a lot more for less power – and cost to customers – than previous models. It’s called TPU chips, not GPUs, because the smaller chips were called Tensor.

But Google’s chips are not a direct threat to Nvidia, perhaps not. Like other major cloud providers, including Microsoft and AmazonGoogle is using chips to augment Nvidia’s performance offering in its infrastructure. It’s not replacing Nvidia. In fact, Google promised that its cloud will have the latest device from Nvidia, Vera Rubin, available later this year.

One day the hyperscalers building their own AI chips (which includes Amazon, Microsoft, and Google) may grow and need Nvidia less, as businesses move their AI needs to their clouds and deploy their applications to these chips.

However, as things stand today, it is not profitable to bet against Nvidia. As a well-known expert in the chip market Patrick Moorhead wrote jokingly on Xpredicted that Google’s TPU could be bad news for Nvidia (and Intel) back in 2016 when the search giant launched the first one. Nvidia is now a company with a market cap of about $5 trillion, which means that the prediction did not come true in time.

If all goes according to Nvidia’s plan, Google’s growth as an AI cloud provider could mean more business for the chip maker at least, even if most of the work is running on Google’s hardware.

In fact, Google also says it has agreed to work with Nvidia to develop computer engineers that will allow Nvidia-based systems to perform better in its cloud. In particular, the two tech giants are working on a new computer technology called Falcon, which Google created and opened in 2023 under the godfather of all open data center hardware organizations, the Open Compute Project.