Stable Diffusion generates images based on the prompt provided, and while it creates reliable results, it struggles to get the details right, irrespective of how detailed the prompt gets.
For this, VAE (Variational Autoencoder) models serve a great purpose to enhance the overall quality while improving the details, particularly facial structures and backgrounds. In this guide, we will help you get started with the VAE model by downloading and using them on Stable Diffusion with ease.
VAE in Stable Diffusion
- Start by navigating to Hugging Face on your browser and sign in with your credentials. Once you’re on the home, navigate to the search bar and enter VAE.
- By doing so, you will find multiple VAE models and pick the model that fits your requirements based on the available results. However, if you’re unsure, download the VAE model from here.
- After you download the model file, move or copy it to the VAE – models folder in Stable Diffusion WebUI (stable-diffusion-webui\models\VAE).
- If you have done it right, you will find the model added to the VAE folder along with the Put VAE here text document.
- With the VAE model moved to its assigned folder, restart your device for the changes to save and reflect.
- After a successful reboot, open Stable Diffusion and then generate an image using txt2img. For demonstration, we generated the image of a woman and noticed inconsistencies around her eyes.
- To address inconsistencies, open the Settings on Stable Diffusion WebUI. Next, navigate to the options on the left, click on Stable Diffusion, and then VAE.
- From the multiple settings that are displayed, click on the drop-down below SD VAE and select the VAE that you downloaded earlier.
- The VAE model is now selected, and to test if it enhances details, we have generated another image of a woman and were able to get better results without inconsistencies around eyes or teeth.
Here is a side-by-side comparison of two different images with and without VAE:
Wrapping Up
Congratulations on learning how to get reliable images without inconsistencies on Stable Diffusion!
VAE models do an exceptional job of getting the details right. However, it isn’t always the same case due to improper prompts and incompatibility with the checkpoint used. Hence, we suggest you download and use the right VAE while experimenting with the prompt.
Related Reading
How To Use Stable Diffusion for NSFW
