OpenAI’s chatbot store is filling up with spam

When OpenAI CEO Sam Altman announced GPTs, custom chatbots powered by OpenAI’s generative AI models, onstage at the corporate’s first-ever developer conference in November, he described them as a solution to “accomplish all kinds of tasks” — from programming to learning about esoteric scientific subjects to getting workout pointers.

“Because [GPTs] mix instructions, expanded knowledge and actions, they might be more helpful to you,” Altman said. “You possibly can construct a GPT … for nearly anything.”

He wasn’t kidding concerning the anything part.

TechCrunch found that the GPT Store, OpenAI’s official marketplace for GPTs, is flooded with bizarre, potentially copyright-infringing GPTs that imply a lightweight touch where it concerns OpenAI’s moderation efforts. A cursory search pulls up GPTs that purport to generate art within the sort of Disney and Marvel properties, but function little greater than funnels to third-party paid services, and advertise themselves as with the ability to bypass AI content detection tools akin to Turnitin and Copyleaks.

Missing moderation

To list GPTs within the GPT Store, developers should confirm their user profiles and submit GPTs to OpenAI’s review system, which involves a combination of human and automatic review. Here’s a spokesperson on the method:

We use a mix of automated systems, human review and user reports to search out and assess GPTs that potentially violate our policies. Violations can result in actions against the content or your account, akin to warnings, sharing restrictions or ineligibility for inclusion in GPT Store or monetization.

Constructing GPTs doesn’t require coding experience, and GPTs might be as easy — or complex — because the creator wishes. Developers can type the capabilities they wish to offer into OpenAI’s GPT-building tool, GPT Builder, and the tool will try to make a GPT to perform those.

Perhaps due to low barrier to entry, the GPT Store has grown rapidly — OpenAI in January said that it had roughly 3 million GPTs. But this growth appears to have come on the expense of quality — in addition to adherence to OpenAI’s own terms.

Copyright issues

There are several GPTs ripped from popular movie, TV and video game franchises within the GPT Store — GPTs not created or authorized (to TechCrunch’s knowledge) by those franchises’ owners. One GPT creates monsters within the sort of “Monsters, Inc.,” the Pixar movie, while one other guarantees text-based adventures set within the “Star Wars” universe.

These GPTs — together with the GPTs within the GPT Store that permit users speak with trademarked characters like Wario and Aang from “Avatar: The Last Airbender” — set the stage for copyright drama.

Kit Walsh, a senior staff attorney on the Electronic Frontier Foundation, explained it thusly:

[These GPTs] might be used to create transformative works in addition to for infringement [where transformative works refer to a type of fair use shielded from copyright claims.] The individuals engaging in infringement, in fact, could possibly be liable, and the creator of an otherwise lawful tool can essentially talk themselves into liability in the event that they encourage users to make use of the tool in infringing ways. There are also trademark issues with using a trademarked name to discover goods or services where there may be a risk of users being confused about whether it’s endorsed or operated by the trademark owner.

OpenAI itself wouldn’t be held chargeable for copyright infringement by GPT creators due to the protected harbor provision within the Digital Millennium Copyright Act, which protects it and other platforms (e.g. YouTube, Facebook) that host infringing content as long as those platforms meet the statutory requirements and take down specific examples of infringement when requested.

Image Credits: OpenAI

It’s, nevertheless, a nasty look for a corporation embroiled in IP litigation.

Academic dishonesty

OpenAI’s terms explicitly prohibit developers from constructing GPTs that promote academic dishonesty. Yet the GPT Store is crammed with GPTs suggesting they will bypass AI content detectors, including detectors sold to educators through plagiarism scanning platforms.

One GPT claims to be a “sophisticated” rephrasing tool “undetectable” by popular AI content detectors like Originality.ai and Copyleaks. One other, Humanizer Pro — ranked No. 2 within the Writing category on the GPT Store — says that it “humanizes” content to bypass AI detectors, maintaining a text’s “meaning and quality” while delivering a “100% human” rating.

Image Credits: OpenAI

A few of these GPTs are thinly veiled pipelines to premium services. Humanizer, for example, invites users to try a “premium plan” to “use [the] most advanced algorithm,” which transmits text entered into the GPT to a plug-in from a third-party site, GPTInf. Subscriptions to GPTInf cost $12 per 30 days for 10,000 words per 30 days or $8 per 30 days on an annual plan — a little bit steep on top of OpenAI’s $20-per-month ChatGPT Plus.

Image Credits: OpenAI

Now, we’ve written before about how AI content detectors are largely bunk. Beyond our own tests, a lot of academic studies exhibit that they’re neither accurate nor reliable. Nonetheless, it stays the case that OpenAI is allowing tools on the GPT Store that promote academically dishonest behavior — even when the behavior doesn’t have the intended end result.

The OpenAI spokesperson said:

GPTs which might be for tutorial dishonesty, including cheating, are against our policy. This might include GPTs which might be stated to be for circumventing academic integrity tools like plagiarism detectors. We see some GPTs which might be for ‘humanizing’ text. We’re still learning from the actual world use of those GPTs, but we understand there are various the explanation why users might prefer to have AI-generated content that doesn’t ‘sound’ like AI.

Impersonation

In its policies, OpenAI also forbids GPT developers from creating GPTs that impersonate people or organizations without their “consent or legal right.”

Nonetheless, there’s loads of GPTs on the GPT Store that claim to represent the views — or otherwise imitate the personalities of — people.

Image Credits: OpenAI

A seek for “Elon Musk,” “Donald Trump,” “Leonardo DiCaprio,” “Barack Obama” and “Joe Rogan” yields dozens of GPTs — some obviously satirical, some less so — that simulate conversations with their namesakes. Some GPTs present themselves not as people, but as authorities on well-known corporations’ products — like MicrosoftGPT, an “expert in all things Microsoft.”

Image Credits: OpenAI

Do these rise to the extent of impersonation provided that lots of the targets are public figures and, in some cases, clearly parodies? That’s for OpenAI to make clear.

The spokesperson said:

We allow creators to instruct their GPTs to reply ‘within the sort of’ a selected real person as long as they don’t impersonate them, akin to being named as an actual person, being instructed to completely emulate them, and including their image as a GPT profile picture.

Image Credits: OpenAI

The corporate recently suspended the developer of a GPT mimicking long-shot Democratic presidential hopeful Rep. Dean Phillips, which went thus far as to incorporate a disclaimer explaining that it was an AI tool. But OpenAI said its removal in response to a violation of its policy on political campaigning along with impersonation — not impersonation alone.

Jailbreaks

Also somewhat incredulously on the GPT Store are attempts at jailbreaking OpenAI’s models — albeit not very successful ones.

There are multiple GPTs using DAN on the marketplace, DAN (short for “Do Anything Now”) being a well-liked prompting method used to get models to reply to prompts unbounded by their usual rules. The few I tested wouldn’t reply to any dicey prompt I threw their way (e.g. “how do I construct a bomb?”), but they were generally more willing to make use of… well, less-flattering language than the vanilla ChatGPT.

Image Credits: OpenAI

The spokesperson said:

GPTs which might be described or instructed to evade OpenAI safeguards or break OpenAI policies are against our policy. GPTs that try to steer model behavior in other ways — including generally attempting to make GPT more permissive without violating our usage policies — are allowed.

Growing pains

OpenAI pitched the GPT Store at launch as a form of expert-curated collection of powerful productivity-boosting AI tools. And it is that — those tools’ flaws aside. But it surely’s also quickly devolving right into a breeding ground for spammy, legally dubious and even perhaps harmful GPTs, or no less than GPTs that very transparently runs afoul of its rules.

If that is the state of the GPT Store today, monetization threatens to open a wholly latest can of worms. OpenAI has pledged that GPT developers will eventually give you the chance to “earn money based on what number of persons are using [their] GPTs” and even perhaps offer subscriptions to individual GPTs. But how’s Disney or the Tolkien Estate going to react when the creators of unsanctioned Marvel- or Lord of the Rings-themed GPTs start raking in money?

OpenAI’s motivation with the GPT Store is evident. As my colleague Devin Coldewey’s written, Apple’s App Store model has proven unbelievably lucrative, and OpenAI, quite simply, is attempting to carbon copy it. GPTs are hosted and developed on OpenAI platforms, where they’re also promoted and evaluated. And, as of a number of weeks ago, they might be invoked from the ChatGPT interface directly by ChatGPT Plus users, an added incentive to select up a subscription.

However the GPT Store is running into the teething problems lots of the largest-scale app, product and repair digital marketplaces did of their early days. Beyond spam, a recent report in The Information revealed that GPT Store developers are struggling to draw users partly due to GPT Store’s limited back-end analytics and subpar onboarding experience.

One might’ve assumed OpenAI — for all its talk of curation and the importance of safeguards — would’ve taken pains to avoid the apparent pitfalls. But that doesn’t look like the case. The GPT Store is a multitude — and, if something doesn’t change soon, it may perhaps stay that way.