SambaNova Systems has unveiled and started shipping its second-generation DataScale system, the DataScale SN30. This new system is powered by the Cardinal SN30 RDU (Reconfigurable Data Unit), and SambaNova claims it delivers a 6x performance improvement on certain AI workloads compared to systems using Nvidia A100 GPUs. The hardware is available for shipment starting today.
The Cardinal SN30 RDU, built on TSMC’s 7nm process, contains 86 billion transistors and can deliver 688 teraflops at bfloat16 precision. Marshall Choy, SambaNova’s senior vice president for product, mentioned that while the company is not releasing detailed chip specs, they focus on delivering systems and services using the chips rather than selling them separately. Each SN30 system includes eight SN30 RDUs and other upgrades, with a major focus on memory capacity. Choy pointed out that the SN30 offers 12.8 times more memory than Nvidia’s DGX A100 system—1 terabyte compared to 80 gigabytes—doubling both compute and memory compared to the previous generation.
SambaNova’s software stack, SambaFlow, is integrated with enterprise features like Kubernetes support for orchestrating containerized and virtualized models and applications. Choy emphasized that the new DataScale SN30 is easy to deploy—customers just need to roll it into place, plug it in, and it’s ready to go. Customers can purchase the systems directly or access them through SambaNova’s “Dataflow-as-a-Service” model, which can be deployed via cloud services or on-premises private clouds, depending on customer preferences for privacy and security.
SambaNova is positioning the DataScale SN30 against Nvidia’s systems, claiming it outperforms an eight-socket DGX A100 system by six times when training a 13-billion parameter GPT-3 model. In comparison to the earlier DataScale SN10, the new system offers 2-6x better performance.
The company has revealed initial customers such as Argonne National Laboratory and Lawrence Livermore National Laboratory (LLNL), both known for adopting new AI technologies. LLNL’s CTO, Bronis de Supinski, highlighted that the DataScale system will significantly improve performance and productivity at their facility.
Gartner analyst Chirag Dekate noted that as users dive deeper into AI, particularly foundation models, SambaNova’s advancements in accelerator performance and Dataflow-as-a-Service are making it easier for organizations to experiment with complex models. Dekate added that the new offering lowers the entry barriers for those exploring AI innovations.
SambaNova is focused on the growing demand for foundation models—large AI models capable of performing various specialized tasks. Choy explained that businesses are moving beyond predictive analytics and machine learning to adopt foundation models, which he sees as essential in the future, much like high-speed internet and mobile apps are today. For example, a large bank that once relied on thousands of fine-tuned BERT models for specific tasks is now using foundation models like GPT-3, which provide equal or better accuracy.
Choy also pointed out that building and deploying foundation models, such as GPT-3, is challenging and mainly managed by a few companies, including OpenAI, AWS, Meta, and SambaNova. The company’s Dataflow-as-a-Service already offers a GPT model, and enhancements are underway, with plans to add a vision-oriented foundation model soon.
SambaNova has raised $1.1 billion at a $5 billion valuation and employs over 500 people. The company will be present at SC22 in Dallas this November and has several speaking slots at the AI Hardware Summit, which runs this week.
