
Broadcom is one of many companies developing custom AI accelerators. So it’s a big deal that the company showed off an AI computing ASIC with optical connectivity at Hot Chips 2024. This was likely done for a customer project. Broadcom makes custom AI accelerators for hyperscalers. The company also showed off co-packaged optics and silicon photonics, which is something we’d like to see.
We’re live on Hot Chips 2024 this week so please excuse any typos.
Broadcom AI Computing ASIC with Optical Attachment Details Revealed at Hot Chips 2024
Co-packaged optics has been a hot topic for a long time.

One of the big challenges in interconnect and networking is the electrical I/O reach through the PCB. While some designs use cables externally from the switch chip to the optical cage, the end goal still seems to be optics. However, the NVIDIA NVLink NVL72 is a great example of a copper connection rather than an optical connection.

Silicon photonics enables optical modules with a reduced component count.

Broadcom now uses silicon photonics and packaged optics not just for switches but also for scale-up computing.

In case you were wondering, the switch in the photo is a Broadcom Tomahawk 5 Bailly.

Here’s the schematic for CPO. The key difference between Intel’s old Silicon Photonics and Broadcom is that Broadcom uses pluggable lasers. Lasers fail, so having pluggable lasers allows for better maintainability. Similarly,

Broadcom has a version of the CPO (co-packaged optics) component in the Tomahawk 5 slide shown earlier in this presentation.

The Tomahawk 4 “Humboldt” was a first-generation system.

This is very similar to Broadcom’s OCP 2022 CPO demo.

This is the Tomahawk 4 implementation from Silicon Photonics.

The new version is the Tomahawk 5 Bailly, a 51.2T Ethernet switch in the same class as the Marvell Teralynx 10 51.2T 64-port 800GbE switch we introduced recently.

In the new version, Broadcom has improved the packaging, resulting in a more stable platform for optical interconnects, as FOWLP is used in mobile chips and more.

Here are the steps to create a chip with an engine:

Here’s a cross-section of the optical engine: Broadcom says this is a more scalable way to manufacture and integrate the optical engine.

A few months ago, Broadcom pitched Bially the idea of having a 128-port 400G optical module. These optical modules include a light source, so it would have been more accurate for Broadcom to have included the light source.

According to Broadcom, 800G modules consume 13-15W of power. With CPO and removing DSP complexity etc, this power drops to less than 4.8W.

One of the key challenges is not only achieving the optical connection but making it work without error.

Now, Broadcom says it is further optimizing its solution, a step to show vendors that it is on par with the quality they get with traditional pluggable optics.

In a 51.2T switch, the optical network plug consumes a lot of power. Using co-packaged optics reduces the total power. The result is approximately 30% power savings.

The next step will be to use similar techniques to combine compute ASICs with co-packaged optics. Here you can see CoWoS with HBM, compute ASIC, and optical chiplets packaged together.

The GPU Attach looks a bit more advanced than the chips mentioned above, with more HBM and compute tiles, but still, the idea is that you can get 64 links out of the chip and plug them directly into a switch.

Keeping the optics away from the xPU is also important because it keeps the optical engine away from hot compute. This may not seem like a big deal, but it is important to ensure the optics work reliably.

Currently we have 64x 100G devices and 2 per 12.8T. What’s insane about this is that the density goes from 12.8T to 102.4T, which is an almost insane amount of bandwidth.

Broadcom is showcasing its use of bidirectional optical technology for high-radix networks, where transmission and reception occur at different wavelengths.

This bidirectional optical approach is attractive because it lowers fiber costs.

There are plenty here.
Final Words
Optical connectivity has to come at some point. Intel announced its Silicon Photonics Connector in 2022, but still uses copper interconnects on its chips, even though it had plans to replace its HBM stack with optical connectivity via Lightbender next year in 2025 (Intel has shut down Lightbender). Broadcom does a lot of optical networking thanks to its networking business, and now ships packaged optical switches. The move to optical I/O will be a big deal for the AI space as companies look to build larger packages.
This was definitely one of the most amazing presentations this year.