In this guide, you will learn how to create beautiful and unique QR codes with Stable Diffusion and the ControlNet plugin.
Table of Contents
Prerequisites
You will need to following tools:
- Stable Diffusion (GitHub)
- Stable Diffusion Model (CivitAI) <– Any other model should work fine
- ControlNet Plugin (GitHub)
- ControlNet QR Model (CivitAI) <– Any other QR code model should work fine
- QR Code Generator (34QR) <– Any other generator should work fine
Install Stable Diffusion
- Create a folder where you want to install Stable Diffusion
- Open a command prompt and navigate to this folder
- Clone the Stable Diffusion Repository into your folder
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git- Download your preferred model from CivitAI
- Move the model to the “models/Stable-diffusion” directory in your Stable Diffusion folder
- Run webui.bat (Windows) or webui.sh (Linux / MacOS) and wait for the installation to complete. This will take a while
- Once the script has finished you will see the IP and port you need to connect to with your web browser (Most likely 127.0.0.1:7680)
Install ControlNet
- Within the Stable Diffusion WebUI go to Extensions –> Install from URL
- Enter the following URL and press install
https://github.com/Mikubill/sd-webui-controlnet.git- Once installed, select the “Installed” tab and “Apply and restart UI”
- Restart the Stable Diffusion instance by terminating and restarting the application from the terminal
- Move the ControlNet model into the “models/ControlNet” folder and refresh the model list (top left corner)
Creating a baseline QR code
Stable diffusion cannot encode text into QR codes on its own, so you need to supply a basic QR code. Any QR code generator should work. I suggest 30% error correction to get the best results. Save the QR code to a location of your choice.
Setting up parameters
These parameters can be used as a baseline to build upon.
- Open the img2img tab and set the QR code as the source image
- Set the following parameters as a baseline
- Resize mode: Just resize
- Sampling method: DPM++2M Karras
- Sampling steps: 20
- Scroll down until you see a collapsed “ControlNet” menu and open it
- Set your QR code as the ControlNet image
- Set the following ControlNet parameters as a baseline
- Enable: True
- Pixel Perfect: True
- Control Type: Tile
- Model: Your QR code model
- Control Weight: 1.3
- Starting Control Step: 0.1
At this point, all that is left to do is to prompt Stable Diffusion. From my testing, I found that certain prompts work better as the subject blends more naturally with the QR code style. I also recommend starting with short queries and working your way up as you are beginning to see good results.
Test, Tweak, Repeat
Not every output will result in a working QR code. Only about 1/5 of the outputs tend to work. Depending on your models you may find that different settings work better for you. These are some settings you should play with:
- Sampling steps: Refers to the number of times the generated image is improved iteratively, with higher values taking longer but producing better / more detailed results.
- Control Weight: Refers to the weight of the ControlNet’s influence. The higher the value, the more readable the QR codes become. This in turn will reduce “uniqueness” as the codes are more visible.
- Resolution: Increasing the resolution can help with retaining the QR code better but may also introduce too many details.
- Model: Different models can return different results. You can try different models for Stable Diffusion as well as ControlNet to find what suits you.
I advise you to test the QR codes with a third-party app as well as the built-in reader of your camera app. During my tests, I noticed that my iPhone’s camera app would not recognize some of the QR codes, while the QR-Code app did.
You should also test your outputs by moving the camera closer or further away from the code as this may make some of them readable.
If you found a style that works for you, replace the Stable Diffusion source image with a good-looking prototype. This will result in the next codes to look more like your template.
The creative approach
If working QR codes are not important to you and you just like the way the output looks, you can get some very creative results. All you need to do is to reduce the control weight and, if desired, change the input picture for Stable Diffusion, to get some very nice results.
Conclusion
Stable Diffusion shows how versatile it can be, once again. These QR codes are definitely an eye-catcher if you get a working one. It was a lot of fun to experiment with this topic, even with all the headaches it created.
