Mistral releases Codestral, its first generative AI model for code

Mistral, the French AI startup backed by Microsoft and valued at $6 billion, has released its first generative AI model for coding, dubbed Codestral.

Like other code-generating models, Codestral is designed to assist developers write and interact with code. It was trained on over 80 programming languages, including Python, Java, C++ and JavaScript, explains Mistral in a blog post. Codestral can complete coding functions, write tests and “fill in” partial code, in addition to answer questions on a codebase in English.

Mistral describes the model as “open,” but that’s up for debate. The startup’s license prohibits using Codestral and its outputs for any business activities. There’s a carve-out for “development,” but even that has caveats: The license goes on to explicitly ban “any internal usage by employees within the context of the corporate’s business activities.”

The rationale might be that Codestral was trained partly on copyrighted content. Mistral didn’t confirm or deny this within the blog post, however it wouldn’t be surprising; there’s evidence that the startup’s previous training datasets contained copyrighted data.

Codestral may not be well worth the trouble, in any case. At 22 billion parameters, the model requires a beefy PC as a way to run. (Parameters essentially define the skill of an AI model on an issue, like analyzing and generating text.) And while it beats the competition in line with some benchmarks (which, as we all know, are unreliable), it’s hardly a blowout.

While impractical for many developers and incremental when it comes to performance improvements, Codestral is bound to fuel the controversy over the wisdom of counting on code-generating models as programming assistants.

Developers are definitely embracing generative AI tools for a minimum of some coding tasks. In a Stack Overflow poll from June 2023, 44% of developers said that they use AI tools of their development process now while 26% plan to soon. Yet these tools have obvious flaws.

An evaluation of greater than 150 million lines of code committed to project repos over the past several years by GitClear found that generative AI dev tools are leading to more mistaken code being pushed to codebases. Elsewhere, security researchers have warned that such tools can amplify existing bugs and security issues in software projects; over half of the answers OpenAI’s ChatGPT gives to programming questions are improper, in line with a study from Purdue.

That won’t stop firms like Mistral and others from attempting to monetize (and gain mindshare with) their models. This morning, Mistral launched a hosted version of Codestral on its Le Chat conversational AI platform in addition to its paid API. Mistral says it’s also worked to construct Codestral into app frameworks and development environments like LlamaIndex, LangChain, Proceed.dev and Tabnine.