QR codes can have up to 30% redundancy. So 30% of the area could be covered, destroyed, or nonsensical, and it's still valid. They are pretty resilient.
It honestly is a more impressive achievement for the QR code protocol than for SD. These images were mangled pretty hard but QR codes have robust built-in error correction that can handle it. I think something like 30% of the QR code can be corrupted and it will still work.
To SD's credit though the images look like relatively normal QR codes if you shrink them down to a thumbnail size so it managed to be remarkably creative while not touching the low-frequency data needed by the protocol.
QR Codes are also scaled up in density depending on the size of the input. This URL is VERY small, which makes it possible for the black and white areas to be quite large.
I just tried this with a longer URL and it failed completely. Wouldn't even recognize it as a QR code (presumably because enough of the checksum failed.)
Yes. Here's a non-AI product that works on the same principle https://uniqr.us/. It uses the picture you upload and draw a QR over it. What folks don't realize is that there's actually techniques you can use to control where the white/black dots end up on QR codes (given that the URL is not too long), and with some math trickery, you can place them in a way that gives the picture extra clarity.
But what the AI is doing here is not only controlling the dots to match the picture, but also bending the details of the picture (brighter bits and darker bits) to match the QR's requirements on the image.
Even if it does get mangled, I'm fairly certain you could simply layer the original code on top of the generation afterwards to make certain that it's functional.
This goes pretty deep, but sort of makes sense to how the error correction works in a QR code. They can be generated so that X% of the code (5, 10, 20, 50+%) is illegible and it'll still read. Fascinating. https://youtu.be/X8jsijhllIA
QR-codes can reconstruct themselves, based on the information given. For example, QR codes with pictures in them, like whatsapp QR codes, only work because of this. The same thing is happening here, I think
Depending on how well you can hide the QR code, it might make for some really fun puzzle solving stuff, and might be useful for hiding information in plain sight.
Does anyone have a workflow then on how to go about training a new model for this sort of image generation?
Ah, I guess the article has some explanation of the training process...
The training of ControlNet has high requirements on data volume and computing power. The training data volume recorded in the paper ranges from 80,000 to 3 million, and the training time can reach 600 A100 GPU hours. Fortunately, the author provided a basic training script, and HuggingFace also implemented Diffusers.
In the previous JAX Sprint, we were lucky enough to use Google TPU v4 to complete the training of 3 million images very quickly. It's a pity that the event is over, and we returned to the laboratory's A6000/4090, training a version of 100,000 images, and the learning rate is very large, just to appear "Sudden Convergence " as soon as possible.
I guess it's not feasible to reproduce on my local machines, lol. Darn.
I have the feeling this will not be made open source... Yesterday I found that page as well and added it to my favourites. Now it's gone. I'm really interested in how this is done. I hope they will release it.
Canny is surely not the way as it only detects outlines. scribble_xdog seems to work kinda when you push the XDoG threshold all the way, but it comes nothing close to OP's results.
This. THIS. This is art. This is beautiful. This means something. The fact that it works makes it genius. I feel like I’m looking at the start of a new generation in art, watching something new and fresh and incredible. The blending of formats, of expectations, is just amazing. Absolute props OP
Was JUST thinking about this yesterday. QR codes have really needed more unique brand customization. In the future, the QR element will be 100% invisible. You'll just scan a logo and go...
ControlNet 0: preprocessor: lineart_standard (from white bg & black line), model: control_v11p_sd15_lineart, weight: 1, starting/ending: (0, 1), resize mode: Just Resize, pixel perfect: True, control mode: ControlNet is more important
ControlNet 1: preprocessor: canny, model: control_v11p_sd15_canny, weight: 0.85, starting/ending: (0, 1), resize mode: Just Resize, pixel perfect: True, control mode: ControlNet is more important
ControlNet 2: preprocessor: depth_leres++, model: control_v11f1p_sd15_depth, weight: 1.9, starting/ending: (0, 1), resize mode: Just Resize, pixel perfect: True, control mode: ControlNet is more important.
If X/Y/Z can use multi controlnet, it should be possible to make better images, different styles of QR codes, which can help generate better images. I used TXT2IMG to generate similar images, by 2 controlnet, I generated about 500 images, and finally got more than 30 QR codes that can be successfully scanned with cell phones. thanks the author!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I just tried a user-friendly QR Code AI Generator Tool. With this tool, I created some stunning QR code images that worked perfectly. Here it's: QR Generation, Reinvented - Taptok
I imagine a piece of architecture or art in real life, where if you look at it from the right perspective it forms a QR code. Perhaps it only becomes visible when the shadows of certain lighting conditions form the dark areas.
Anyone ever played The Witness? Very similar vibes.
year is 2077 look at peculiar art on some wall camera in your eye recognizes it as a 2D barcode format and reads the data you get a computer brain virus
The results are not as a good as the nhciao images. Some feature of Automatic1111 don't work with hf-diffusers as for example the controlnet step control and controlnet preprocessor. I will try to figure to add them to hf-diffusers and to figure out the best settings so everyone could make similar result from home.
If you have any advice feel free to comment or make a pull requests / open a post on Community.
PS: I will not keep it in public for very long time
I love this :) ... Finally aome good stuff that isn't scantily clad perfectly sculpted women, i saw another post with "normal" looking women and it was AWESOME too... The perfection of human imperfection made it so easy in the eyes
Has anyone replicated this yet. I tried using control net and was only able to make the most trivial cosmetic changes before damaging the integrity of the qr code beyond readability.
Would it be better to start with a high res qr code or a low res one? Anyone found a reliable workflow?
I've been messing around with QR codes and ControlNet today and one thing I've learned is that the registration marks (the large square symbols in the corners of the op images) need to be basically unchanged. It's seems there is a higher tolerance for changing the rest of the QR code, but not much.
For example, this variation did not work until I recolored the round registration marks back to black and white. Now it works.
Also, I started with a lower resolution/simpler QR code from QR Code Generator
Nice, I could kind of get this far but not as pretty. I was trying to incorporate an image. I think that QR codes are designed to be robust, but left to it's own devices SD sprinkles enough errors randomly all over it to corrupt it. Tonight I'm going to try inpainting on a low-res code.
This is an attack vector. You could hide them in wall paper or some other kind of innocuous art and then people wouldn't notice them but their devices would.
I really want to know how to replicate this; the specific thing I want to do is create a set of artistic kitchen backsplash tiles with QR codes embedded in art (like the third one) that link to recipes that we use regularly.
I'm completely noob in QR codes but do these really follow the QR code protocol? 30% error correction that people argue is only valid if the 70% leftovers are "normal", right? In fact, we've already found differences in readability among QR code readers in this thread, which must not happen in a valid QR code. I think implementations of QR code readers shall be complete but not necessarily sound, which implies that even if a picture can be read as a QR code, it might not follow the protocol and therefore there is no guarantee that other readers can read it.
Amazing. Longer text to decode may not worked as the checksum needs to get bigger as the error correction must take a huge part of the QR in these. Still amazing!
2
u/nhciao Jun 08 '23
Sorry for the lateness, check out our article (in Chinese): https://mp.weixin.qq.com/s/i4WR5ULH1ZZYl8Watf3EPw
…and follow us for updates:
https://github.com/ciaochaos
https://huggingface.co/ioclab