​Microsoft joins forces with Qingbei to promote rStar-Math technology: small models counterattack mathematical problems, surpassing OpenAI!

Microsoft recently announced its new rStar-Math technology. This innovative reasoning method can be applied to small language models (SLMs), significantly improving their performance on mathematical problems, and even surpassing OpenAI's o1-in some cases. preview model. This technology is still in the research stage, and related research papers have been published on arXiv.org and were jointly completed by eight authors from Microsoft, Peking University, and Tsinghua University.

In tests, rStar-Math technology was applied to multiple small open source models, including Microsoft's Phi-3 mini model, Alibaba's Qwen-1.5B (1.5 billion parameter model) and Qwen-7B (7 billion parameter model). The test results show that the performance of all participating models has improved. In the MATH benchmark test, rStar-Math even surpassed OpenAI's previous most advanced model.

The research team plans to open relevant code and data on Github, although it is still under internal review and has not yet been made public. The community has expressed great interest in this technology, with many members praising its stepwise reasoning method combined with Monte Carlo Tree Search (MCTS), believing that this innovation has broad application prospects in fields such as geometric proofs and symbolic reasoning.

The core of rStar-Math is the use of Monte Carlo tree search, a method that simulates human "deep thinking" and helps small models evolve themselves by gradually refining solutions to mathematical problems. The researchers not only simply applied MCTS, but also required the model to simultaneously provide natural language reasoning steps and Python code during the output process. Such requirements promote efficient training of models.

After four rounds of self-evolution, rStar-Math has achieved remarkable results in multiple benchmark tests. In the MATH benchmark test, the accuracy of the Qwen2.5-Math-7B model jumped from 58.8% to 90.0%, surpassing OpenAI's o1-preview. In the American Invitational Mathematics Competition (AIME), the model solved 53.3% of the problems, ranking in the top 20% of high school competitors.

In recent years, artificial intelligence innovation has mainly relied on the continuous increase of model parameters. However, the high costs that come with it have made people question the sustainability of this expansion. Microsoft demonstrated the potential of small models through rStar-Math, emphasizing the direction of high performance. The release of this technology demonstrates that specialized small-scale models can serve as a powerful alternative to larger systems, providing mid-sized organizations and academic researchers with cutting-edge capabilities without the huge financial and environmental burdens.

Paper entrance: https://arxiv.org/pdf/2501.04519