Google’s DeepMind creates generative AI model with fact checker to crack unsolvable math problem

Google LLC’s DeepMind artificial intelligence research unit claims to have cracked an unsolvable math problem using a big language model-based chatbot equipped with a fact-checker to filter out useless outputs.

Through the use of a filter, DeepMind researchers say the LLM can generate tens of millions of responses, but only submit those that might be verified as accurate.

It’s a milestone achievement, as previous DeepMind breakthroughs have generally relied on AI models that were specifically created to resolve the duty in hand, resembling predicting weather or designing latest protein shapes. Those models were trained on very accurate and specific datasets, which makes them quite different from LLMs resembling OpenAI’s GPT-4 or Google’s Gemini.

Those LLMs are trained on vast and varied datasets, enabling them to perform a wide selection of tasks and speak about almost any subject. However the approach carries risks, as LLMs are vulnerable to so-called “hallucinations,” which is the term for producing false outputs.

Hallucinations are an enormous problem for LLMs. Gemini, which was only released this month and is alleged to be Google’s most capable LLM ever, has already shown it’s vulnerable, inaccurately answering fairly easy questions resembling who won this yr’s Oscars.

Researchers consider that hallucinations might be fixed by adding a layer above the AI model that verifies the accuracy of its outputs before passing them onto users. But this sort of safety net is hard to construct when LLMs have been trained to debate such a wide selection of topics.

At DeepMind, Alhussein Fawzi and his team members created a generalized LLM called FunSearch, which is predicated on Google’s PaLM2 model. They added a fact-checking layer, called an “evaluator.” On this case, FunSearch has been geared to solving only math and computer science problems by generating computer code. In line with DeepMind, this makes it easier to create a fact-checking layer, because its outputs might be rapidly verified.

Although the FunSearch model continues to be vulnerable to hallucinations and generating inaccurate or misleading results, the evaluator can easily filter them out, and make sure the user only receives reliable outputs.

“We predict that perhaps 90% of what the LLM outputs is just not going to be useful,” Fawzi said. “Given a candidate solution, it’s very easy for me to let you know whether this is definitely an accurate solution and to guage the answer, but actually coming up with an answer is de facto hard. And so mathematics and computer science fit particularly well.”

In line with Fawzi, FunSearch is capable of generate latest scientific knowledge and concepts, which is a brand new milestone for LLMs.

The researchers tested its abilities by giving it an issue, plus a really basic solution in source code, as an input. Then, the model generated a database of latest solutions that were checked by the evaluator for his or her accuracy. Essentially the most reliable of those solutions are then fed back into the LLM as inputs, along with a prompt asking it to enhance on its ideas. In line with Fawzi, by doing it this manner, FunSearch produces tens of millions of potential solutions that eventually converge to create probably the most efficient result.

When tasked with mathematical problems, FunSearch writes computer code that may find the answer, slightly than attempting to tackle it directly.

Fawzi and his team tasked FunSearch with finding an answer to the cap set problem, which involves determining patterns in points, where no three points make a straight line. Because the variety of points grows, the issue becomes vastly more complex.

Nonetheless, FunSearch was capable of create an answer consisting of 512 points across eight dimensions, which is larger than any human mathematician has managed. The outcomes of the experiment were published within the journal Nature.

Although most persons are unlikely ever to come back across the cap set problem, let alone try to solve it, it’s a vital achievement. Even one of the best human mathematicians don’t agree on one of the best technique to solve this challenge. In line with Terence Tao, a professor on the University of California, who describes the cap set problem as his “favorite open query,” FunSearch is a particularly “promising paradigm” since it may potentially be applied to many other math problems.

FunSearch proved as much when tasked with the bin-packing problem, where the goal is to efficiently place objects of various sizes into the least variety of containers as is feasible. Fawzi said FunSearch was capable of find solutions that outperform one of the best algorithms created to resolve this particular problem. Its results could have significant implications in industries resembling transport and logistics.

FunSearch can also be notable because, unlike with other LLMs, users can actually see the way it goes about generating its outputs, meaning they will learn from it. This sets it aside from other LLMs, where the AI is more akin to a “black box.”

Image: bedneyimages/Freepik