In the video 'NVIDIA Removed This. Tenstorrent Put It Back.' on the Alex Ziskind channel, the host reviews the Tenstorrent Wormhole N300, a unique non-NVIDIA AI hardware card. He compares its performance, architecture, and power efficiency against NVIDIA's RTX 5080 and AMD's RX 7900, highlighting the critical role of software maturity in AI hardware.
Channel Alex Ziskind
Duration 13:00
Views Unavailable
Published Unavailable
The Wormhole N300
The Tenstorrent Wormhole N300 is a $1,400 AI card featuring two chips and 24 GB of memory, but no video outputs. After overcoming initial setup and firmware challenges, it achieved 30 to 31 tokens per second running Llama 3.1 8B locally, with throughput scaling to 164 tokens per second under concurrent requests.
Tensix Core Architecture
Unlike NVIDIA's CUDA cores, the Wormhole N300 uses a grid of Tensix cores, each containing five open-source RISC-V CPUs that act as traffic cops. This design streams data through a dedicated matrix engine with its own chip memory, preventing data from constantly hammering slower system memory.
Restoring Card-to-Card Networking
Tenstorrent includes QSFP networking ports on the Wormhole N300, a feature NVIDIA stripped from consumer GPUs like the RTX 4090 and 5090 to reserve for $30,000 datacenter chips. These ports allow multiple cards to connect directly and work together as one, bypassing the slower PCIe bus.
NVIDIA Head-to-Head Comparison
In a direct comparison running an 8-bit quantized Llama 3.1 8B model, the NVIDIA RTX 5080 outperformed the Wormhole N300 by a factor of three, achieving 93 tokens per second. Even in performance-per-watt efficiency, the RTX 5080 won because its high token throughput outweighed its higher power draw.
The Software Bottleneck
While both Tenstorrent and AMD (tested with the RX 7900) have highly capable silicon, they lag behind NVIDIA because of software. NVIDIA's massive lead is driven by 15 years of CUDA development, meaning the success of alternative hardware depends entirely on how quickly their software stacks can catch up.
While NVIDIA GPUs remain the superior choice for plug-and-play local AI performance and efficiency today due to their mature CUDA software ecosystem, Tenstorrent's Wormhole N300 offers a compelling, open-source alternative for tinkerers willing to bet on its unique architecture and rapidly improving software stack.
Source www.youtube.com ↗
Mission Squad