Microsoft officially open sourced the super powerful small model Phi-4, and its performance test surpassed GPT-4o and Llama-3.1

Microsoft recently released a small language model called Phi-4 on the Hugging Face platform. This model has only 14 billion parameters, but it has performed well in multiple performance tests, surpassing many well-known models, including OpenAI's GPT. -4o and other similar open source models such as Qwen2.5 and Llama-3.1.

In a previous test at the American Mathematics Competition AMC, Phi-4 scored 91.8 points, significantly better than competitors such as Gemini Pro1.5 and Claude3.5Sonnet. What is even more surprising is that this small parameter model achieved a high score of 84.8 in the MMLU test, fully demonstrating its powerful reasoning and mathematical processing capabilities.

Unlike many models that rely on organic data sources, Phi-4 uses innovative methods to generate high-quality synthetic data, including techniques such as multi-agent prompting, instruction inversion, and self-correction. These methods greatly enhance Phi-4's reasoning and problem-solving capabilities, allowing it to handle more complex tasks.

Phi-4 uses a decoder-only Transformer architecture that supports context lengths up to 16k, making it ideal for processing large input data. Approximately 10 trillion tokens are used in its pre-training process, combining synthetic data with strictly screened organic data to ensure excellent performance in benchmark tests such as MMLU and HumanEval.

Features and benefits of Phi-4 include: Compactness and efficiency for consumer-grade hardware; Inference capabilities that exceed previous generations and larger models in STEM-related tasks; Support for fine-tuning with diverse synthetic data sets for easy Meet the needs of specific areas. In addition, Phi-4 provides detailed documentation and API on the Hugging Face platform to facilitate integration by developers.

In terms of technological innovation, the development of Phi-4 mainly relies on three pillars: multi-agent and self-correction technology to generate synthetic data, post-training enhancement methods such as rejection sampling and direct preference optimization (DPO), and strictly filtered training data, This ensures that overlapping data with the benchmark is minimized, improving the model’s generalization ability. In addition, Phi-4 utilizes Key Tag Search (PTS) to identify important nodes in the decision-making process, thereby optimizing its ability to handle complex reasoning tasks.

With the open source of Phi-4, developers' expectations have finally come true. The model can not only be downloaded from the HuggingFace platform, but also supports commercial use under the MIT license. This open policy has attracted the attention of a large number of developers and AI enthusiasts, and HuggingFace’s official social media also congratulated it, calling it “the best 14B model in history.”

Model entrance: https://huggingface.co/microsoft/phi-4