Overview
Overall, although the ASICs developed by other third-party companies have their own highlights, they still face a rather severe market environment. On the one hand, these companies need to catch up with Nvidia's interconnect technology in hardware. On the other hand, their software stacks are far from being as mature as the CUDA environment. Therefore, these companies still face significant challenges in the process of catching up with Nvidia's GPUs.
- Groq: Groq's chip uses a bufferless, fully deterministic VLIW architecture. It excels in low-latency scenarios cost-effectively but relies on SRAM without HBM for large models, needing multi-chip interconnect. Challenges include scalability costs and performance trade-offs. Mitigation strategies involve better KV Cache management, SRAM for weights, and leveraging process tech for scalability.
- SambaNova: SN40L chip has a multi-core compute-storage array. It optimizes storage and compute units, with PCU and PMU architectures enhancing precision and matrix ops. Internal unit collaboration and external storage options boost performance for large language models, improving throughput and efficiency.
- Tenstorrent: Blackhole AI chip, based on RISC-V multi-core with Metalium tool, has many compute units and SRAM. Its multi-core design offers flexibility and energy efficiency. Single-card performance matches A100, though market entry is delayed.
- Cerebras: WSE chip, made from a whole wafer, is huge with numerous compute units and large SRAM. It uses near-memory computing, performing well in large-scale AI but facing efficiency challenges as compute and storage aren't fully integrated.