The short version
- Build a test set of at least six image types, at sizes people really upscale.
- Run every tool at the same scale factor, on default settings first and then on its recommended best setting.
- Run creative tools at least three times per image, and keep every run.
- Crop matched regions such as eyes, text, foliage and edges, and view them at 100% or more.
- Judge in blind pairs with sides randomised, against a stated goal: closest to the original, or most pleasing.
- Record wins per category with notes on artefacts, speed and cost. Don’t average everything into one score.
1. Build the test set
| Category | Typical input | What it exposes |
|---|---|---|
| AI renders | 1024 px Midjourney, SDXL or Flux output | Whether creative detail looks coherent or repetitive |
| Phone portraits | A 1–3 MP crop of a face | Identity drift, waxy skin, invented eyelashes |
| Products with text | A packshot with a small label | Melted or misspelled lettering, distorted logos |
| Old scans and JPEGs | A 600 ppi scan or a heavily compressed web image | Whether compression blocks get “sharpened” into texture |
| Line art and pixel art | Anime frames, sprites, UI icons | Edge handling, colour banding, unwanted texture |
| Landscapes and foliage | Trees, grass, water | Tiling, repeated patterns, “oil painting” look |
Include at least one image where you know the ground truth: take a large original, downscale it by 4× yourself, upscale it back, and compare against the original. It’s the only way to tell real recovery from plausible invention.
2. Control the settings
- Use the same scale factor for every tool. 4× is the common denominator.
- Run default settings first, because that’s what most people use, then each vendor’s documented best-quality setting.
- Run creative tools three times or more. The variance between runs is part of the result.
- Note the cost and time for each output, as shown by the tool.
3. Judge blind, in pairs
Compare outputs two at a time, with labels hidden and sides randomised. People reliably prefer whatever they know is the expensive tool. Our free blind A/B tool runs several rounds over random crops of the same pair and reveals the tally at the end. Decide the goal before you look: “closest to the original” and “most pleasing” often disagree, and both are legitimate.
4. Look at matched crops, not thumbnails
View at 100% or 200% on the regions where upscalers differ: eyes and teeth, small text, hair against sky, foliage, fabric weave and hard diagonal edges. A full image scaled to fit the screen throws away most of the extra pixels, which is the whole thing you’re testing.
5. Record what you saw
| Criterion | Question to answer |
|---|---|
| Fidelity | Is it still the same person, product or place? |
| Detail | Is the texture recovered, or generated? Does generated texture make sense? |
| Artefacts | Halos, ringing, waxy skin, repeated tiles, seams, colour shifts? |
| Text | Is every character still correct? |
| Consistency | Do several runs of the same image agree? |
| Cost and time | Per image at your target size |
Report wins per category rather than one blended number. A tool that wins on AI art and loses on product labels is useful to know about. “8.7/10” tells you nothing.
Common ways comparisons mislead
- Different output sizes. A 4× result compared with a 2× one upscaled in the browser.
- Re-compression. Results saved as JPEG at different qualities. Compare PNGs.
- Vendor sample images. They are chosen to flatter the tool. Use your own.
- One run of a creative model. You might have seen its best or its worst result.
Not sure which kind of upscaler you need in the first place? Read creative vs faithful upscaling.