ExploreToM
ExploreToM generates diverse psychological theory datasets using A* search for complex story structures enhancing AI models in understanding human cognition
English
English
ExploreToM is a framework developed by Facebook Research to generate diverse and challenging psychological theory data at scale for reinforcement of training and evaluation of large language models (LLMs). The framework utilizes the A* search algorithm to generate complex story structures and novel, diverse and reasonable scenarios on custom domain-specific languages to test the limits of LLMs.
Demand population:
"The target audience is researchers, developers and educational institutions, who can use the data generated ExploreToM to train and evaluate models of psychological theoretical reasoning, thereby improving the ability of artificial intelligence to understand human psychological states."
Example of usage scenarios:
The researchers trained a psychological theoretical reasoning model using data generated by ExploreToM .
Educational institutions use this framework to create teaching cases that help students understand psychological theories.
Developers use the ExploreToM framework to test and improve their chatbots or virtual assistants.
Product Features:
Generate story background: Use the story_context_generator.py script to generate story background.
Perform A* Search: Perform A* Search through story_structure_searcher.py script to generate complex story structures.
Fill the generated story: Fill the generated story with the story_structure_infiller.py script.
Statistical analysis: Statistical analysis of the generated data through the compute_statistics.py script.
Functional test: Run tests_belief_tracker.py and tests_story_structure_infiller.py for functional tests.
Model Loading: Use VLLM (large language model) to load and run the model.
Tutorials for use:
1. Install the necessary Python environment and dependencies.
2. Use story_context_generator.py to generate story background.
3. Perform A* search through story_structure_searcher.py to generate complex story structure.
4. Use story_structure_infiller.py to fill the generated story.
5. Run compute_statistics.py to perform statistical analysis on the generated data.
6. Functional testing is performed through tests_belief_tracker.py and tests_story_structure_infiller.py.
7. Load and use the VLLM model as needed.
