Google Reports AI Server Memory Costs Exceed 75%, Unveils Dual Hardware-Software Strategy

At the SEMICON Taiwan 2026 Memory Summit, Google Cloud's Senior Director of Supply Chain Infrastructure, Nikhil Cherian, stated that AI computing has shifted from compute-constrained to memory-constrained. High-performance memory now accounts for over 75% of AI server hardware bill-of-materials costs. To address bottlenecks in capacity, bandwidth, and power, Google is implementing a dual strategy. Hardware-wise, it has introduced the TPU 8i for low-latency inference and the TPU 8t for large-scale training. The TPU 8i features 288 GB of high-bandwidth memory