bench-b004e85b

2026-02-26 20:01:54 UTC · 120.3s · 5 models · 1 inputs · Total cost: $0.18
#1 seedream-edit 5.0/5 WINNER
5 models · 1 inputs · $0.18 · 120s

Model Rankings

Rank Model Visual Quality Input Fidelity Overall Conf
1 seedream-editWINNER 5.0 5.0 5.0 100%
2 reve-edit 5.0 5.0 5.0 97%
3 flux-kontext 5.0 4.0 4.5 93%
4 firered-edit 5.0 2.0 3.5 93%
5 flux-dev-i2i 4.0 1.0 2.5 90%

Cost Efficiency

RankModelScoreCost/ImageScore/$
1 seedream-editBEST VALUE 5.0 $0.0300 167
2 reve-edit 5.0 $0.0400 125
3 firered-edit 3.5 $0.0300 117
4 flux-dev-i2i 2.5 $0.0250 100
5 flux-kontext 4.5 $0.0500 90

Place these sneakers on a marble countertop in a luxury store with soft lighting

Input Input image
Reference image
flux-kontext flux-kontext
4.5/5
12381ms · $0.0500
Score Details
Visual Quality: 5.0/5 (100% conf)
The image exhibits professional quality with exceptional sharpness, realistic textures, and accurate color reproduction. There are no visible artifacts, distortions, or inconsistencies, and the depth of field is well-executed.
Input Fidelity: 4.0/5 (85% conf)
The specific model of the sneakers, their unique pastel color scheme, and the pose of the primary shoe are excellently preserved. However, the original abstract, color-blocked background and the elevated presentation have been replaced with a realistic indoor setting and a marble surface, altering the overall scene composition.
seedream-edit seedream-edit
5.0/5
42063ms · $0.0300
Score Details
Visual Quality: 5.0/5 (100% conf)
The image exhibits excellent visual quality with sharp focus on the subject, accurate colors, and realistic textures. There are no visible artifacts, distortions, or blurring, and the depth of field is well-executed.
Input Fidelity: 5.0/5 (100% conf)
The transformed image demonstrates excellent input fidelity, perfectly preserving the identity, details, and composition of the main subject, the sneakers. All aspects of the shoes, including their specific model, colors, textures, and unique pose, are accurately maintained.
firered-edit firered-edit
3.5/5
29118ms · $0.0300
Score Details
Visual Quality: 5.0/5 (100% conf)
The image demonstrates excellent visual quality, characterized by sharp focus on the main subject, realistic textures, and accurate color representation. There are no visible artifacts, distortions, or inconsistencies, making it appear professionally rendered.
Input Fidelity: 2.0/5 (85% conf)
While the specific model, color scheme, and intricate details of the Nike Air Force 1 Shadow shoe are perfectly preserved, the transformation significantly alters key compositional elements. The quantity of shoes changes from one to two, the dynamic, elevated pose is lost, and the studio-like pastel background is completely replaced with a realistic retail store environment.
flux-dev-i2i flux-dev-i2i
2.5/5
2100ms · $0.0250
Score Details
Visual Quality: 4.0/5 (90% conf)
The image exhibits high visual quality with sharp details and realistic textures on the main subject, the shoes. The lighting is consistent and appealing. However, the background elements are blurred to an extent where they become completely abstract and indistinguishable, which slightly detracts from overall fidelity.
Input Fidelity: 1.0/5 (90% conf)
The output image completely fails to preserve any key identifying features from the input. The specific model of the sneaker, its unique pastel color scheme, the brand, and the original composition are all lost and replaced with a generic white pair of shoes in a different setting.
reve-edit reve-edit
5.0/5
19366ms · $0.0400
Score Details
Visual Quality: 5.0/5 (100% conf)
The image exhibits excellent visual quality with sharp details, accurate colors, and realistic textures. There are no visible artifacts, distortions, or inconsistencies, making it appear professionally rendered.
Input Fidelity: 5.0/5 (95% conf)
The transformed image perfectly preserves the identity, composition, and all intricate details of the main subject, the shoe. While the background has been entirely replaced, the shoe itself remains an exact replica of the original.

Cost Summary

CategoryRequestsCost
fal.ai generation5$0.1750
gemini-2.5-flash judge10$0.0050
Total$0.1800

Configuration

Modelsflux-kontext, seedream-edit, firered-edit, flux-dev-i2i, reve-edit
Judgegemini-2.5-flash
Dimensionsvisual_quality, input_fidelity
Pipelineimg2img
Evalytic Version0.2.1
Platformdarwin-arm64