AI method that can think without language

For the first time, researchers have uncovered these "black box" reasoning processes using "thought chain" prompts to allow large language models (LLMs) to reason step by step. However, research by Meta and the University of California, San Diego points out that thinking in natural language may become a bottleneck in the development of AI. They proposed a new method called COCONUT (Continuous Chain of Thoughts), which redefines the way artificial intelligence reasons.

Language bottleneck challenge

The traditional thinking chain method requires the AI ​​model to convert each step of reasoning into natural language, leading to the following problems:

1. A large number of generated tags are just language filler (such as "therefore", "next step", etc.) and do not provide substantive inferential value.

2. Models are forced to use specific vocabulary at key decision points, resulting in reduced efficiency.

3. Grammatical coherence takes up computing resources and makes it difficult for the model to focus on practical problem solving.

Research has found that when humans deal with complex reasoning tasks, the language center of the brain often remains "quiet", but existing AI systems are forced to translate reasoning into text, forming a so-called "language bottleneck."

COCONUT’s Innovation: Reasoning with Neural Spaces

COCONUT overcomes the limitations of traditional methods by letting AI think in the underlying neural space. Its core concept is similar to the human problem-solving process:

1. Absorption issues : taking in information (e.g. reading rules).

2. Silent Thinking : The mind explores possibilities without having to translate them into words.

3. Share the solution : Finally articulate the results in words.

In COCONUT, the AI ​​model can freely switch between "language mode" and "neural mode":

• Language mode : Used to understand a question or provide an answer.

• Neural Patterns : Complex Reasoning in a Continuous Space Unconstrained by Language.

Training process: Gradually unlocking potential COCONUT training is divided into three stages:

1. Basic stage : The model uses traditional thinking chain reasoning to establish basic understanding.

2. Transition phase : Gradually remove verbal expressions and encourage the model to develop internal neural thinking patterns.

3. Balance stage : achieve seamless switching between language and neural patterns.

Experimental results: The potential of COCONUT

Research has verified COCONUT's capabilities in a variety of reasoning tasks:

• Mathematics word problems (GSM8k) : COCONUT accuracy is 34.1%, slightly lower than traditional thinking chain (42.9%).

• Logical reasoning (ProntoQA) : COCONUT achieves an accuracy of 99.8%, which is higher than the traditional method (98.8%), and only uses 9 markers (CoT requires 92.5).

• Complex planning (ProsQA) : COCONUT far exceeds traditional methods (77.5%) with an accuracy of 97%, using only 14.2 markers (traditional methods require 49.4).

Towards more efficient artificial intelligence

COCONUT redefines the possibilities of artificial intelligence reasoning. By getting rid of language limitations, it demonstrates efficient and flexible reasoning capabilities, laying the foundation for building more natural and powerful artificial intelligence systems. This breakthrough method reveals the future development direction of AI from language-driven to continuous thinking.