Creativity is a trait that AI critics say is more likely to remain the preserve of humans for the foreseeable future. But a large-scale study finds that leading generative language models can now exceed the typical human performance on linguistic creativity tests.
The query of whether machines may be creative has gained latest salience lately because of the rise of AI tools that may generate text and pictures with each fluency and magnificence. While many experts say true creativity is unimaginable without lived experience of the world, the increasingly sophisticated outputs of those models challenge that concept.
In an effort to take a more objective have a look at the difficulty, researchers on the Université de Montréal, including AI pioneer Yoshua Bengio, conducted what they are saying is the most important ever comparative evaluation of machine and human creativity thus far. The team compared outputs from leading AI models against responses from 100,000 human participants using a standardized psychological test for creativity and located that the perfect models now outperform the typical human, though they still trail top performers by a major margin.
“This result could also be surprising—even unsettling—but our study also highlights an equally necessary remark: even the perfect AI systems still fall in need of the degrees reached by essentially the most creative humans,” Karim Jerbi, who led the study, said in a press release.
The test at the center of the study, published in Scientific Reports, is often known as the Divergent Association Task and involves participants generating 10 words with meanings as distinct from each other as possible. The upper the typical semantic distance between the words, the upper the rating.
Performance on this test in humans correlates with other well-established creativity tests that give attention to idea generation, writing, and inventive problem solving. But crucially, it is usually quick to finish, which allowed the researchers to check a much larger cohort of humans over the web.
What they found was striking. OpenAI’s GPT-4, Google’s Gemini Pro 1.5 and Meta’s Llama 3 and Llama 4, all outperformed the typical human. Nevertheless, after they measured the typical performance of the highest 50 percent of human participants, it exceeded all tested models. The gap widened further after they took the typical of the highest 25 percent and top 10 percent of humans.
The researchers desired to see if these scores would translate to more complex creative tasks, so additionally they got the models to generate haikus, movie plot synopses, and flash fiction. They analyzed the outputs using a measure called Divergent Semantic Integration, which estimates the variety of ideas integrated right into a narrative. While the models did relatively well, the team found that human-written samples were still significantly more creative than AI-written ones.
Nevertheless, the team also discovered they may boost the AI’s creativity with some easy tweaks. The primary involved adjusting a model setting called temperature, which controls the randomness of the model’s output. When this was turned all the best way up on GPT-4, the model exceeded the creativity scores of 72 percent of human participants.
The researchers also found that fastidiously tuning the prompt given to the model helped too. When explicitly instructed to make use of “a method that relies on various etymology,” each GPT-3.5 and GPT-4 did higher than when given the unique, less-specific task prompt.
For creative professionals, Jerbi says the persistent gap between top human performers and even essentially the most advanced models should provide some reassurance. But he also thinks the outcomes suggest people should take these models seriously as potential creative collaborators.
“Generative AI has above all develop into an especially powerful tool within the service of human creativity,” he says. “It’ll not replace creators, but profoundly transform how they imagine, explore, and create—for many who select to make use of it.”
Either way, the study adds to a growing body of research that’s raising uncomfortable questions on what it means to be creative and whether it’s a uniquely human trait. Given the strength of feeling around the difficulty, the study is unlikely to settle the matter, however the findings do mark one in every of the more concrete attempts to measure the query objectively.