Meta-enhanced fine-tuning: a new paradigm for improving the inference ability of large language models | CMU Research

In the field of artificial intelligence, large language models (LLMs) are constantly evolving. Recently, Carnegie Mellon University (CMU) and HuggingFace researchers have proposed a new method called "Meta Reinforcement Fine-Tuning" (MRT). This method aims to optimize the computational efficiency of large language models during testing, especially when solving complex inference problems.

Research shows that existing large language models often consume too much computing resources during the inference process, and the goal of MRT is to enable the model to achieve more efficient answer discovery within a given computational budget. This method splits the output of the large language model into multiple fragments to achieve a balance between exploration and utilization. Through meticulous learning of training data, MRT enables the model to use known information and explore new problem-solving strategies when facing unknown problems.

In the study, experiments by the CMU team showed that the model achieved significant improvements on multiple inference benchmarks after fine-tuning using MRT. In comparison with traditional results reward reinforcement learning (GRPO), MRT has 2 to 3 times accuracy and is 1.5 times more efficient in token use. This means that MRT can not only improve the model's inference ability, but also reduce the consumption of computing resources, thus giving it more advantages in practical applications.

In addition, researchers also proposed how to effectively evaluate the effectiveness of existing inference models, laying the foundation for future research. This achievement not only demonstrates the potential of MRT, but also points out the direction for the application of large language models in more complex application scenarios.

Through such innovations, the research teams of CMU and HuggingFace are undoubtedly at the forefront of pushing AI technology, empowering machines with stronger reasoning capabilities, and laying a solid foundation for achieving smarter applications.

Project address: https://cohenqu.github.io/mrt.github.io/