Deep|Optics: Scale-In Moves Optics Inside the Tray, and the Bandwidth Opportunity Could 10x Scale-Up

Core conclusion

Optics are moving inside the tray, and the memory tier it targets is the largest bandwidth tier in the system, roughly ten times the scale-up bandwidth per GPU and about one hundred times the scale-out. The capital is already moving, with $2 billion each from NVIDIA into Lumentum and Coherent, $3.25 billion from Marvell for Celestial AI, and a revenue ceiling of up to $120 billion on the Google and Marvell warrant. Scale-in, in this note, means die-to-die and chip-to-memory optical links at millimeters to about a meter, which Lumentum's CEO on August 27 called "the thing to watch over the next 12 to 18 months". The word is spreading with split meanings since NVIDIA uses it for something else, and Section 3 sorts out who means what. The kicker is Section 7, where our checks point to the largest scale-in exploration running at Google, built around memory pooling, and the warrant's disclosed scope, "memory interface controllers and near-memory compute", is what we believe to be its public paper trail. The work is early, and first-generation products start well below the ceiling.

1. Scale-In's Coming-Out Week

At an investment conference on August 27, Lumentum CEO Michael Hurlston said, "I think the thing that is going to be significant is the scale-in, and that's the thing to watch over the next 12 to 18 months... Are you going to see it actually go in-tray and serve this high bandwidth connectivity between memory and between GPUs?" He was describing optical links inside trays. Scale-in was also featured in imec's slides on the first day of Semicon Taiwan, in a talk on 3D integrated optics for chip-to-chip interconnect.