Firefly, Adobe’s family of generative AI models, doesn’t have the perfect popularity amongst creatives.
The Firefly image generation model particularly has been derided as underwhelming and flawed in comparison with Midjourney, OpenAI’s DALL-E 3, and other rivals, with a bent to distort limbs and landscapes and miss the nuances in prompts. But Adobe is attempting to right the ship with its third-generation model, Firefly Image 3, releasing this week throughout the company’s Max London conference.
The model, now available in Photoshop (beta) and Adobe’s Firefly web app, produces more “realistic” imagery than its predecessor (Image 2) and its predecessor’s predecessor (Image 1) due to a capability to grasp longer, more complex prompts and scenes in addition to improved lighting and text generation capabilities. It should more accurately render things like typography, iconography, raster images and line art, says Adobe, and is “significantly” more proficient at depicting dense crowds and other people with “detailed features” and “a wide range of moods and expressions.”
For what it’s price, in my transient unscientific testing, Image 3 does seem like a step up from Image 2.
I wasn’t in a position to try Image 3 myself. But Adobe PR sent a number of outputs and prompts from the model, and I managed to run those self same prompts through Image 2 on the internet to get samples to match the Image 3 outputs with. (Have in mind that the Image 3 outputs could’ve been cherry-picked.)
Notice the lighting on this headshot from Image 3 in comparison with the one below it, from Image 2:
From Image 3. Prompt: “Studio portrait of young woman.”
Same prompt as above, from Image 2.
The Image 3 output looks more detailed and lifelike to my eyes, with shadowing and contrast that’s largely absent from the Image 2 sample.
Here’s a set of images showing Image 3’s scene understanding at play:
From Image 3. Prompt: “An artist in her studio sitting at desk looking pensive with tons of paintings and ethereal.”
Same prompt as above. From Image 2.
Note the Image 2 sample is fairly basic in comparison with the output from Image 3 when it comes to the extent of detail — and overall expressiveness. There’s wonkiness happening with the topic within the Image 3 sample’s shirt (across the waist area), however the pose is more complex than the topic’s from Image 2. (And Image 2’s clothes are also a bit off.)
A few of Image 3’s improvements can little question be traced to a bigger and more diverse training data set.
Like Image 2 and Image 1, Image 3 is trained on uploads to Adobe Stock, Adobe’s royalty-free media library, together with licensed and public domain content for which the copyright has expired. Adobe Stock grows on a regular basis, and consequently so too does the available training data set.
In an effort to ward off lawsuits and position itself as a more “ethical” alternative to generative AI vendors who train on images indiscriminately (e.g. OpenAI, Midjourney), Adobe has a program to pay Adobe Stock contributors to the training data set. (We’ll note that the terms of this system are reasonably opaque, though.) Controversially, Adobe also trains Firefly models on AI-generated images, which some consider a form of information laundering.
Recent Bloomberg reporting revealed AI-generated images in Adobe Stock aren’t excluded from Firefly image-generating models’ training data, a troubling prospect considering those images might contain regurgitated copyrighted material. Adobe has defended the practice, claiming that AI-generated images make up only a small portion of its training data and undergo a moderation process to make sure they don’t depict trademarks or recognizable characters or reference artists’ names.
After all, neither diverse, more “ethically” sourced training data nor content filters and other safeguards guarantee a wonderfully flaw-free experience — see users generating people flipping the bird with Image 2. The true test of Image 3 will come once the community gets its hands on it.
Latest AI-powered features
Image 3 powers several recent features in Photoshop beyond enhanced text-to-image.
A brand new “style engine” in Image 3, together with a brand new auto-stylization toggle, allows the model to generate a wider array of colours, backgrounds and subject poses. They feed into Reference Image, an option that lets users condition the model on a picture whose colours or tone they need their future generated content to align with.
Three recent generative tools — Generate Background, Generate Similar and Enhance Detail — leverage Image 3 to perform precision edits on images. The (self-descriptive) Generate Background replaces a background with a generated one which blends into the prevailing image, while Generate Similar offers variations on a specific portion of a photograph (an individual or an object, for instance). As for Enhance Detail, it “fine-tunes” images to enhance sharpness and clarity.
If these features sound familiar, that’s because they’ve been in beta within the Firefly web app for at the least a month (and Midjourney for for much longer than that). This marks their Photoshop debut — in beta.
Speaking of the online app, Adobe isn’t neglecting this alternate path to its AI tools.
To coincide with the discharge of Image 3, the Firefly web app is getting Structure Reference and Style Reference, which Adobe’s pitching as recent ways to “advance creative control.” (Each were announced in March, but they’re now becoming widely available.) With Structure Reference, users can generate recent images that match the “structure” of a reference image — say, a head-on view of a race automotive. Style Reference is basically style transfer by one other name, preserving the content of a picture (e.g. elephants within the African Safari) while mimicking the style (e.g. pencil sketch) of a goal image.
Here’s Structure Reference in motion:
Original image.
Transformed with Structure Reference.
And Style Reference:
Original image.
Transformed with Style Reference.
I asked Adobe if, with all of the upgrades, Firefly image generation pricing would change. Currently, the most cost effective Firefly premium plan is $4.99 per thirty days — undercutting competition like Midjourney ($10 per thirty days) and OpenAI (which gates DALL-E 3 behind a $20-per-month ChatGPT Plus subscription).
Adobe said that its current tiers will remain in place for now, together with its generative credit system. It also said that its indemnity policy, which states Adobe pays copyright claims related to works generated in Firefly, won’t be changing either, nor will its approach to watermarking AI-generated content. Content Credentials — metadata to discover AI-generated media — will proceed to be robotically attached to all Firefly image generations on the internet and in Photoshop, whether generated from scratch or partially edited using generative features.
