Rank
Creator
model
Bradley-Terry
Elo
Wins
Matches
1
ai logo
Black Forest Labs
Flux 1 pro
1135.041057.308410501557622
2
ai logo
Black Forest Labs
Flux 1.1 pro
1082.651040.13496062953979
3
ai logo
xAI
Aurora
1035.381023.89250286491566
4
ai logo
Google
Imagen 3
1019.211018.15455736918268
5
ai logo
Runway
Frames
990.081007.60206445414628
6
ai logo
OpenGVLab
Lumina
969.121000.42102679201610
7
ai logo
OpenAI
Dall-E 3
952.00993.327139671464775
8
ai logo
Stability.ai
Stable Diffusion 3
939.14988.355921151219975
9
ai logo
Midjourney
Midjourney 5.2
893.75970.326423011371196
10
ai logo
DeepSeek
Janus 7B
733.80900.532389555453

What is "Bradley-Terry"?

The Bradley-Terry ranking model is a probabilistic model used to predict outcomes in pairwise comparisons. It assigns a strength parameter (reported score) to each item, indicating its likelihood of winning against another. See the wikipedia article for mathematical details.

What do we consider as "Overall preference"?

Here we evaluate the model across all criteria and determine which model has the best overall performance.

All results are directly based on feedback from real human raters. The process of how we came out with results is best described in our blog post.

Examples

Visual examples of the annotators’ preferences

Preference
Which image looks better overall?
flux-1-pro_winner
FLUX.1 [pro]
flux-1-pro_winner
Midjourney
Coherence
Which image feels less weird or unnatural for its style when you look closely? I.e. fewer odd or strange-looking objects or elements
flux-1-pro_winner
FLUX.1 [pro]
flux-1-pro_winner
Midjourney
Alignment
Which image is more aligned with and better adheres to the prompt:
A black and white picture of a white man singing a song
flux-1-pro_winner
FLUX.1 [pro]
flux-1-pro_winner
Midjourney