Roblox launches real-time AI chat rephrasing to filter out banned language

Roblox is introducing a real-time, AI-powered chat rephrasing feature that robotically replaces banned words with more respectful language, the corporate announced on Thursday.

The brand new feature goes beyond Roblox’s current text filter, which simply replaces banned words and phrases with the “#” symbol. But Roblox says that when users encounter strings of “####,” conversations may be disrupted and hard to follow.

Now, as an alternative of simply displaying hash marks, filtered text can be rephrased into more respectful language that’s near the user’s original intent.

For instance, a message that reads “Hurry TF up!” would previously have appeared as “####,” but will now be rephrased to “Hurry up!” Everyone within the chat is notified that the message has been rephrased to maintain the conversation civil, Roblox says.

“Chat is central to how people connect, coordinate, and play on Roblox,” said Rajiv Bhatia, vice chairman of User and Discovery Product at Roblox, in a press release. “Real-time rephrasing helps keep gameplay and conversations heading in the right direction while guiding language toward what’s appropriate. This approach reduces friction in chat while maintaining the standards that help keep our community civil.”

Roblox notes that while rephrasing reduces a few of the disruption within the chat, its safety system stays in effect for more serious behavior. 

The brand new feature is supported in all languages currently available through Roblox’s automatic translation tools.

The corporate can be upgrading the text-filtering system to detect more variations of banned language. Roblox says early results show that the system is recovering at detecting leetspeak, and more sophisticated attempts to bypass filters. This has allowed the corporate to cut back the prevalence of false negatives when sharing or soliciting personal information by 20x.

Thursday’s announcement comes shortly after Roblox introduced mandatory facial verification for access to chats on its platform following a wave of lawsuits over child safety from the attorneys general of Texas, Kentucky, and Louisiana, amongst others. The lawsuits were filed in response to reports that Roblox was exposing young users to dangerous risks, reminiscent of grooming and explicit content.