POINTS-Yi-1.5-9B-Chat

POINTS-Yi-1.5-9B-Chat is a visual language model that integrates the latest visual language model technology and new technologies proposed by WeChat AI. This model has significant innovations in pre-training data set filtering and model soup (Model Soup) technology, which can significantly reduce the size of pre-training data sets and improve model performance. It performs well on multiple benchmarks and is an important advance in the field of visual language models.

Demand group:

"The target audience is researchers, developers and enterprises, especially those professionals who need model training and application in the field of visual language. This product helps users improve model performance and reduce Compute resource consumption and accelerate the research and development process. "

Example of usage scenario:

In the image description task, POINTS-Yi-1.5-9B-Chat is used to generate detailed image descriptions.

In visual question answering tasks, models are used to answer questions related to images.

In the visual instruction execution task, the model performs corresponding operations based on the images and instructions provided by the user.

Product features:

Integrate the latest visual language model technologies such as CapFusion, Dual Vision Encoder and Dynamic High Resolution.

Use perplexity as a metric to filter pre-training datasets, reduce dataset size and improve model performance.

Apply model soup technology to integrate fine-tuned models from different visual instruction adjustment data sets to further improve performance.

Excellent performance in multiple benchmark tests, including MMBench-dev-en, MathVista, HallucinationBench, etc.

Supports Image-Text-to-Text multi-modal interaction, suitable for scenarios that require a combination of vision and language.

Detailed usage examples and codes are provided to facilitate developers to quickly get started and integrate.

Usage tutorial:

1. Install necessary libraries such as transformers, PIL and torch.

2. Import AutoModelForCausalLM and AutoTokenizer, as well as CLIPImageProcessor.

3. Prepare image data, which can be network images or local images. picture.

4. Load the model and word segmenter, specify the model path as 'WePOINTS/POINTS-Yi-1-5-9B-Chat'.

5. Configure generation parameters, such as the maximum number of new tokens, temperature, top_p and number of beams.

6. Use the chat method of the model to pass in parameters such as images, prompts, word segmenters, and image processors.

7. Get the model output and print the results.