Mistral AI, the company behind the open source Mistral, Mathstral, and Codestral language models, just introduced its first multimodal AI model. The new Pixtral 12B can process links and images, alongside text.
Sophia Yang, the head of developer relations at Mistral AI, first announced the new model on Twitter. The GitHub repo and the Mistral AI database on Hugging Face have already been updated with the new Pixtral model.
The Pixtral 12B draws from Nemo 12B (another free language model from Mistral), but builds on it with added image processing capabilities. The “12B” in the name refers to the 12 billion parameters of this model. For comparison, ChatGPT 4 has more than a trillion parameters, so Pixtral is a relatively small model. And while Pixtral technically is multimodal, it’s not quite on par with ChatGPT or Anthropic’s Claude, which also understand voice prompts and documents.
You can chat about images with Pixtral and get useful answers like captions or identify what’s in an image. You can feed it single or multiple image files or image URLs with prompts like “what’s this plant?” or “create a caption for this image.”
Right now, you can download the Pixtral model for free via a torrent magnet link. It’s a 24GB file that you can run locally on supported hardware. Mistral AI provides it under an Apache 2.0 license, which means it’s free for personal and commercial purposes. And developers can modify it in any way. Mistral didn’t disclose details of the training dataset for this model.
Mistral AI has plans to offer Pixtral 12B as an official API in the “Le Platforme” stack, and it’ll also appear in the “Le Chat” chatbot soon, presumably as a free demo with a button for uploading images. Mistral charges a fee to access the APIs, so you’ll likely need a subscription to get the Pixtral API keys.
Source: Twitter
