The open source inference AI model Sky-T1 was launched, and the training cost is less than US$450

NovaSky, a research team at the University of California, Berkeley's Sky Computing Laboratory, released its Sky-T1-32B-Preview inference model on Friday. The model performed well on multiple key benchmarks, rivaling OpenAI's early version of o1, and more. What is striking is its extremely low training cost.

Sky-T1-32B-Preview is the first truly open source inference model. The NovaSky team not only released the model, but also disclosed the data set used to train it and the necessary training code, which means that the model can be replicated from scratch. According to the team in a blog post, "Sky-T1-32B-Preview costs less than $450 to train, demonstrating that advanced inference capabilities can be cost-effectively replicated." Not long ago, training models of equivalent performance would have cost tens of thousands of dollars. Millions of dollars, and today the cost is significantly reduced, mainly due to the application of synthetic training data or training data generated by other models. For example, Palmyra X004, a recently released model by artificial intelligence company Writer, was trained almost entirely on synthetic data and cost just $700,000 to develop.

Inference models, unlike ordinary AI models, are able to effectively fact-check themselves, thus avoiding some common pitfalls. However, it often takes longer for the inference model to arrive at a solution, ranging from seconds to minutes. But its greater reliability in fields such as physics, science and mathematics is a significant advantage.

The NovaSky team revealed that they used Alibaba's QwQ-32B-Preview inference model to generate the initial training data for Sky-T1, then "organized" the data, and used OpenAI's GPT-4o-mini to reconstruct the data into a more user-friendly Format. It takes about 19 hours to train Sky-T1 with 32 billion parameters using 8 Nvidia H100 GPU racks. The number of parameters roughly corresponds to the model's ability to solve the problem.

In performance testing, Sky-T1 outperformed o1's early preview version on MATH500, a set of "competition-grade" math challenges, and also beat o1's on a set of difficult questions from LiveCodeBench, a coding assessment. Preview version. However, Sky-T1 does not perform as well on GPQA-Diamond as the o1 preview version, which contains physics, biology and chemistry-related problems that PhD graduates should master. In addition, OpenAI’s o1GA version is more powerful than the preview version, and OpenAI is expected to release a better-performing inference model o3 in the next few weeks.

Still, the NovaSky team says Sky-T1 is just a starting point for their development of open-source models with advanced inference capabilities. “Going forward, we will focus on developing more efficient models, maintaining strong inference performance, and exploring advanced technologies to further improve the efficiency and accuracy of models during testing,” the team wrote in the post. “Please continue to follow us Progress on these exciting plans. "The emergence of this open source inference model has undoubtedly brought new opportunities and challenges to the field of artificial intelligence, and its future development deserves continued attention.