The outputs of text-to-image models are often quite low resolution (for example Stable Diffusion v1.5 outputs 512x512 images) so we want a way to increase that resolution before sharing them on social media, printing them out, or minting as NFTs. The process to do this is called upscaling or super-resolution and there are many different tools and techniques to achieve it.
Before AI models were common, resampling algorithms like bicubic interpolation and Lancszos resampling were frequently used. They use simple mathematical formulas to interpolate between existing pixels in the image. This can produce blurry results since no new details are introduced.
Using AI techniques, models can be trained on large datasets of high-resolution images to learn how to generate realistic patterns and textures. The AI-based techniques can produce much more convincing and detailed results than traditional methods.
512x512 test image
Repeats pixels to increase resolution
Cubic polynomial in 4x4 neighborhood, sharp edges but can overshoot
Windowed sinc functions for less aliasing and ringing
Papers:
GAN (generative adversarial network) based: Super-Resolution with Deep CNN (2014), SRGAN (2016), ESRGAN (2018), Real-ESRGAN (2021), SwinIR (2021)
Diffusion (denoising) based: SRDiff (2021), Latent Diffusion (2022), Stable Diffusion Upscaler (2022)
Free online tools: (each is available in Visions of Chaos too)
Real-ESRGAN - Colab notebook, HuggingFace space, Replicate
SwinIR - HuggingFace space, Replicate
Latent diffusion - Replicate
Stable Diffusion Upscaler - Colab notebook, HuggingFace space
Much sharper than bicubic with hallucinated textures
Less harsh/contrasty than ESRGAN, better texture
More synthesized details but can be noisy
Fewer color artifacts than latent diffusion upscale but less detail
letsenhance.io - online service, pay for credits or monthly subscription
neural.love - online service, pay as you go credits or monthly subscription
On1 Resize AI - AI photo resizer
Topaz Gigapixel - $99 mac and Windows application
Topaz Photo AI - $199 mac and Windows application, adds extra noise removal, sharpen, face enhance
vance.ai - free online upscaler
Super crunchy texture
Lacking detail, too smooth
Sharp with great details
Resize your image to a higher resolution then divide your image into overlapping tiles with guide lines in Photoshop, export each tile, run img2img to add detail to each tile, use layer mask to seamlessly composite tiles. (Owl bot made May 14, 2022 with Disco Diffusion and custom trained bot model - buy him on objkt to support my AI art addiction!)
You can use the Ultimate Upscale extension for automatic1111 Stable Diffusion Web UI as one way to automate this process. The results have much more detail than you get with the GAN models but you have to watch out for mini version of your prompts being stamped out throughout the image. Lowering the guidance and denoising strengths can help.
So what should you use? It's up to you! Each method has different artifacts and you may want to swap between techniques depending on the piece. Do you want things to be soft and smooth or gritty? I sprung for Topaz Gigapixel and tend to use it first. If I like the results I roll with it otherwise I hop over to a diffusion upscaler or SwinIR to get the job done. Don't be afraid to use multiple tools and composite the results in Photoshop or your image editor of choice. Just experiment and have fun!
