Classic resizing: interpolation
Ordinary resizing (nearest-neighbour, bilinear, bicubic, Lanczos) computes each new pixel as a weighted average of nearby existing pixels. It’s fast and predictable, and it never adds information, so enlargements look soft. Sharper kernels such as Lanczos trade some of that softness for faint “ringing” halos near edges.
Learned super-resolution
Neural super-resolution models learn what high-resolution detail usually looks like from millions of examples. Training is simple in principle: take sharp images, degrade them on purpose (downscale, blur, add noise, JPEG-compress), and train the network to recover the original. At inference time the model applies what it has learned: edges become crisp, textures re-appear, compression blocks disappear.
The first deep-learning approach, SRCNN (Dong et al., 2014), was a three-layer convolutional network. Today’s models are far larger and fall into the families covered in ESRGAN vs diffusion upscalers.
Why the “degradation model” matters
A model only learns to undo the damage it saw in training. Early models were trained on clean bicubic downscales and fell apart on real phone photos and web JPEGs. Real-ESRGAN (Wang et al., 2021) made a big practical leap by training on a deliberately messy, “high-order” chain of blur, noise, resizing and compression. That’s why it copes with real-world images.
Accuracy vs realism: you can’t max both
Blau and Michaeli’s perception–distortion trade-off (CVPR 2018) showed formally that an algorithm can’t simultaneously minimise its distortion (its pixel error against the true image, measured by PSNR or SSIM) and maximise how natural its output looks. Models optimised for PSNR look smooth; models trained with adversarial (GAN) or diffusion objectives look sharper and more realistic, but their detail is partly invented. Every “creativity” slider is a dial along this trade-off.
| Metric | Measures | Tends to reward |
|---|---|---|
| PSNR | Pixel-level error against a reference | Safe, slightly blurry results |
| SSIM | Structural similarity (Wang et al., 2004) | Correct structure and contrast |
| LPIPS | Distance in a deep network’s feature space (Zhang et al., 2018) | Results that look similar to people |
| Blind human preference | Which one people pick | What you actually care about. See our method |
What it can’t do
- Recover information that was never captured. Faces, text and plates a few pixels wide become guesses.
- Undo heavy motion blur or missed focus reliably. Those need deblurring models, and even those have limits.
- Give you a free bigger sensor. A 4× upscale of a phone photo isn’t equivalent to a photo from a higher-resolution camera.
FAQ
Is AI upscaling the same as “enhance” in crime dramas?
No. Super-resolution produces a plausible high-resolution image consistent with the low-resolution one. When the information was never captured, as with a face of a few pixels or a distant number plate, the output is a guess that can look convincing and be wrong.
Why do upscalers say 4× when the file is 16 times bigger?
The scale factor applies to each side. A 4× upscale of 1000 × 1000 px gives 4000 × 4000 px: four times the width, sixteen times the pixels.
What is PSNR and should I care?
Peak signal-to-noise ratio measures how close an output is to a reference image, pixel by pixel. It’s useful in research, but it rewards blur. A soft image that is “on average” right can score higher than a sharp, realistic one. Perceptual metrics such as LPIPS, and blind human comparison, track what people prefer more closely.