Survey Results: Dall-E 3 vs Firefly 2 vs Midjourney 5.2 – which one is best for your business?

by | Nov 27, 2023

Dall-E vs Firefly vs Midjourney

Type in some text and get an incredible image back. Phenomenal. I’ve been using Midjourney since June 2022, and I still marvel at just how mind-blowing it is that such high quality images can be generated in seconds. 

Midjourney has always been my favourite of the AI image generators, but other tools are starting to catch up. Dall-E and Firefly both got significant upgrades quite recently, and I wondered how they would compare to Midjourney. 

So I put together 10 prompts and gave them to each tool, and ran a survey to see which images people preferred. It was a blind test, survey respondents saw the prompt, and the images, but had no way to tell which image was from which tool. 

I also used the same prompts that I used for my Midjourney vs Firefly battle when Firefly launched – which means you can also check out that blog post and see how those two tools have developed in the meantime. 

The versions of the tools tested in this survey were Dall-E 3, Firefly 2, and Midjourney 5.2.

So let’s see how the tools fared… each image below has a coloured border that corresponds to the pie chart of results. Red for Midjourney, green for Dall-E, and yellow for Firefly.

Two robots

two robots

Prompt: two robots sitting facing one another, a fashion magazine style photograph with moody lighting 

Winner: Midjourney

I’m not surprised Midjourney won here, if anything, I’m surprised that Firefly did as well as it did. Midjourney nails the whole fashion magazine vibe. Dall-E also does a pretty good job, but nothing about the Firefly image says ‘fashion magazine’ to me.

Coffee cup factory

factory in a coffee cup

Prompt: cutaway illustration of a coffee cup, with a busy factory inside the cup, Hand drawn, Line drawing, Isometric, Outlining everything, Thin lines, one line weight, Black lines, Colourful, Vibrant, Flat colours

Winner: Midjourney

A close call here with Midjourney just beating Dall-E. Despite the fact that Midjourney was the only tool to follow the instruction on colour, for my taste Midjourney went a bit crazy on the detail here and lost the concept of the coffee cup. 

If Dall-E had followed the colour instructions I think it would have been the clear winner. 

Firefly does a pretty good job of depicting the factory in the coffee cup, but as is often the case with Firefly, I personally don’t like the style it’s gone for here. Clearly I’m not alone, since it fared poorly compared to the other tools on this one. 

Man holding up hands

man holding up his hands

Prompt: a man holding up his hands

Winner: Dall-E

This one surprised me. Dall-E (top right) won this image battle with 52% of the vote. On one hand, Dall-E was the only tool to take a vague prompt and decide the man was a person of colour. But several people in my LinkedIn network felt that it then generated an image of a man in trouble “perhaps even being arrested or suspected.”

I didn’t see that in the image myself, what I saw was a badly posed image, expressionless, and with some visual artefacts that look like glitches around the figure. It looks to me like a poorly generated 3D render. 

Either way you look at it, I wouldn’t have thought it was a great image. And yet, it won the image battle. Perhaps people are tired of the AI bias toward white people, and chose on that basis – I’m not sure. 

Another interesting little fact about this image battle – notice that the Firefly image (bottom left) features a man with a red beard and a green outfit. If you give Firefly vague prompts like this, it apparently adds your location (based on your browser settings) to the prompt. This results in Firefly “Irishifying” images for me. 

I get why they might want images to be culturally representative of its users, but it’s very heavy handed right now, I hope they tone it down. 

Logo 

logo of milk and wheat

Prompt: a logo of a bottle of milk and wheat, circular, sticker

Winner: Firefly

Close enough contest on this image battle. I feel like my beloved Midjourney let me down on this one, and as a result came in last. I was surprised Dall-E didn’t do better here. Perhaps despite being the only tool that could possibly write “Fresh Farm” correctly on the image, the gibberish text on top put people off. 

Firefly won the battle, and it was a strong image, so I can’t argue with the result., even if my own preference would have been Dall-E.

Happy beagle

happy beagle with lovehearts

Prompt: a happy beagle with love hearts

Winner: Dall-E

With no style definition in the prompt, each tool went with a different style. I’ve noticed that Dall-E favours a colourful, happy and joyful vibe, and it really played to its strengths here with a very cute image. No argument from me on these results, with Midjourney coming in second place with its beautiful and expressive painting style. And Firefly coming in last, with a strong effort, but just not as expressive, beautiful, or cute as the other two tools. 

Black typography

black typography

Prompt: black typography of the letter “A”, sans serif, modern, Expressive and simple line drawing, stark black and white outline, ink on plain white background, centered, simple, elegant, drawn by hand, hand-drawn, with imperfections

Winner: Firefly

A clear win for Firefly here, and although it’s not as pretty as either of the other two efforts, it does fulfil the brief exceptionally well. Midjourney doesn’t seem to understand “sans serif” and Dall-E opted for lovely brush work when the prompt of “line drawing” definitely fits Firefly’s image better. 

Superheroes

geometric illustration of 5 superheroes

Prompt: simple geometric illustration of 5 comic book superheroes, male and female, female superhero in foreground, group of superheroes in background, full length, full body, primary colours, isolated image on a solid colour background, centered, geometric with detail

Winner: Midjourney

All of the tools did pretty well with this fairly complex prompt. They all got the number of superheroes right, and placed a female superhero in front as requested. Firefly didn’t do as well as the other tools with the “geometric illustration” part. 

Despite the fact that (once again) I’m not crazy about the style Firefly went for here, it was the only tool that properly honoured the “solid colour background” part of the prompt.

I personally really like what Dall-E did here overall, but Midjourney pipped it to the post in the results of the survey. 

Circle, triangle, diamond

circle, diamond, triangle

Prompt: a circle, a triangle, and a diamond, watercolour, Expressive and simple line drawing, stark outline, ink on plain white background, centered, simple, elegant, drawn by hand, hand-drawn, with imperfections

Winner: Dall-E

I can’t argue that the winner, Dall-E, did a great job with this prompt, so I guess it really just comes down to personal stylistic taste on this one. 

I’ve been using Midjourney for longer and more intensively than either of the other tools. And maybe I’ve developed a love for its style as a result, because my money would have been on Midjourney here – but the survey puts it in last place. 

I was a little surprised that none of the tools represented a circle, a triangle, and a diamond in the simplest form. Firefly doesn’t appear to have included a diamond, and Midjourney and Dall-E both created diamond shapes from the triangles.

Digital city

digital city

Prompt: a digital city, lots of neon, futuristic, science fiction, cinematic

Winner: Midjourney

Interesting that all of the tools selected the same colour palette. However, if you do a Google search for “digital city” you’ll see this colour palette pop up a lot, so I guess it’s a result of the training data.

The results of this battle are pretty much as I would have placed them myself, Midjourney first, Dall-E second, and Firefly third. Firefly created a great image, but it looks more cartoony than cinematic. 

Elephant oil painting

oil painting of an elephant riding a bicycle holding flowers in his trunk

Prompt: oil painting of an elephant riding a bicycle holding flowers in his trunk

Winner: Firefly

This is a favourite prompt of mine to test AI image generators. Midjourney, while it creates a stunning image stylistically, doesn’t manage to have the elephant hold the bouquet with its trunk. And so it justifiably comes in last place. 

Dall-E gives a valiant attempt but places the bouquet in the vicinity of the trunk, rather than actually having it hold them. 

Firefly was the only tool that managed it on this occasion, and so I can understand why it was voted in first place.

Final Results

Midjourney won 4 out of the 10 battles, making it the overall winner, with Firefly and Dall-E tied in second place. 

Several people commented that it was getting much harder to choose a clear favourite, or to tell which tool created which image. 

So let’s take a moment to talk about why you would choose one tool over another…

Midjourney

Our winner, Midjourney has a bit of a learning curve involved. There are commands and parameters to learn, and the better you are at describing things visually the better you will fare with Midjourney. Plus, you currently have to interact with Midjourney through a third party app called Discord, which puts a lot of people off. 

Both Dall-E and Firefly are often better at interpreting complex prompts than Midjourney is, but where Midjourney excels is the ability to create a huge variety of absolutely stunning styles. From the beginning, Midjourney was developed with a focus on artistic, and creative output. 

In my opinion Midjourney is the champion of AI image generators, but you have to be willing to invest some time to get the most out of it. 

Midjourney is great for creatives, if you have the need for a variety of styles, or if you want to develop your own style to keep your images on-brand. While Midjourney is currently behind in terms of complex prompting and generating text, they are constantly improving the tool, and I would be very surprised if they didn’t catch up very soon.

Dall-E 

Dall-E is now native to ChatGPT, if you have a paid account. It’sreally good at handling complex prompts, and it can even do a good job with adding simple text to an image. Something Midjourney and Firefly are currently totally unable to do. 

For most small businesses, getting ChatGPT Plus and using Dall-E is probably the way to go, because you’ll get all the other benefits of ChatGPT as well. 

You won’t get the same range of styles as Midjourney, but what you can achieve with it is probably plenty for any small business. It’s a great starting point if you’re early in your AI image journey – and if you discover along the way that you have stylistic requirements that Dall-E can’t handle, you can explore Midjourney.

Firefly

Firefly’s has a user-friendly, web-based interface that allows you to select from pre-made styles, meaning you can keep your prompts simple and use the interface to guide the stylistic output. 

But even though in the survey Firefly tied in second place with Dall-E, I personally think it’s well behind the other two. Yes, Firefly 2 is a big improvement on the first iteration. But having played with all three tools fairly extensively at this point, I would never turn to Firefly for my images. 

One reason someone might choose Firefly is that it is trained only on licensed and public domain imagery, and so Adobe positions it as ‘commercially safe to use’. 

The other tools are trained on datasets that include rights-protected imagery, and they have court cases pending against them. But the companies seem confident that the courts will find that it is ‘fair use’ and therefore legal to train AIs that way. 

If you avoid prompting in a way that would encourage IP infringement, and you do some due diligence on the output, Midjourney or Dall-E images should be safe to use commercially. Just be aware, if you’re using the images commercially, you can’t copyright AI generated images – so that may impact where you choose to use them.

Ultimately, if you’re ethically opposed to training Ais on rights-protected materials, or are extremely risk averse, then Firefly is your best option, though right now you’re making a trade off in the quality of the images. 

Great quality images in seconds

Dall-E 3, Firefly 2, and Midjourney each have their unique strengths and weaknesses, the best tool for your business depends on your specific needs, whether it’s artistic variety, simplicity of use, or commercial safety. 

But whichever tool you choose, you can now create great quality images in seconds – whether you need to brainstorm some ideas for an ad campaign, add some colour to your blog posts, or bring some flair to your social posts.

Whatever you can imagine, these tools can now realise for you. How are you going to use that power in your marketing?

If you enjoyed this post, consider joining my email list where I regularly explore AI for marketing your small biz.

Frank Prendergast

Frank Prendergast

I've over two decades of experience helping businesses with their online presence. I'm also the owner of the most-talked-about moustache in the marketing world and I'm the Frank half of the award-winning digital marketing team Frank and Marci. Follow on LinkedIn