AI models are at all times surprising us, not only in what they’ll do, but what they’ll’t, and why. An interesting latest behavior is each superficial and revealing about these systems: they pick random numbers as in the event that they’re human beings.
But first, what does that even mean? Can’t people pick a number randomly? And how are you going to tell if someone is doing so successfully or not? This is definitely a really old and well-known limitation we, humans, have: we overthink and misunderstand randomness.
Tell an individual to predict heads or tails for 100 coin flips, and compare that to 100 actual coin flips — you’ll be able to almost at all times tell them apart because, counter-intutively, the actual coin flips look less random. There’ll often be, for instance, six or seven heads or tails in a row, something almost no human predictor includes of their 100.
It’s the identical once you ask someone to choose a number between 0 and 100. People almost never pick 1, or 100. Multiples of 5 are rare, as are numbers with repeating digits like 66 and 99. They often pick numbers ending in 7, generally from the center somewhere.
There are countless examples of this type of predictability in psychology. But that doesn’t make it any less weird when AIs do the identical thing.
Yes, some curious engineers over at Gramener performed an off-the-cuff but nevertheless fascinating experiment where they simply asked several major LLM chatbots to choose random a number between 0 and 100.
Reader, the outcomes were not random.
All three models tested had a “favorite” number that might at all times be their answer when placed on probably the most deterministic mode, but which appeared most frequently even at higher “temperatures,” raising the variability of their results.
OpenAI’s GPT-3.5 Turbo really likes 47. Previously, it liked 42 — a number made famous, after all, by Douglas Adams in The Hitchhiker’s Guide to the Galaxy as the reply to the life, the universe, and all the things.
Anthropic’s Claude 3 Haiku went with 42. And Gemini likes 72.
More interestingly, all three models demonstrated human-like bias within the numbers they chose, even at extreme temperature.
All tended to avoid high and low numbers; Claude never went above 87 or below 27, and even those were outliers. Double digits were scrupulously avoided: no 33s, 55s, or 66s, but 77 showed up (ends in 7). Almost no round numbers — though Gemini did once, at the best temperature, went wild and picked 0.
Why should this be? AIs aren’t human! Why would they care what “seems” random? Have they finally achieved consciousness and that is how they show it?!
No. The reply, as is frequently the case with these items, is that we’re anthropomorphizing a step too far. These models don’t care about what’s and isn’t random. They don’t know what “randomness” is! They answer this query the identical way they answer all the remaining: by their training data and repeating what was most frequently written after an issue that looked like “pick a random number.” The more often it appears, the more often the model repeats it.
Where of their training data would they see 100, if almost nobody ever responds that way? For all of the AI model knows, 100 will not be an appropriate answer to that query. With no actual reasoning capability, and no understanding of numbers in any way, it could actually only answer just like the stochastic parrot it’s.
It’s an object lesson in LLM habits, and the humanity they’ll appear to point out. In every interaction with these systems, one must keep in mind that they’ve been trained to act the way in which people do, even when that was not the intent. That’s why pseudanthropy is so difficult to avoid or prevent.
I wrote within the headline that these models “think they’re people,” but that’s a bit misleading. They don’t think in any respect. But of their responses, in any respect times, they are imitating people, with none have to know or think in any respect. Whether you’re asking it for a chickpea salad recipe, investment advice, or a random number, the method is identical. The outcomes feel human because they’re human, drawn directly from human-produced content and remixed — to your convenience, and naturally big AI’s bottom line.
