Using AI to generate minifigures, part 2
Posted by Huw,
Here's another article by Pawel, an Artificial Intelligence engineer in Poland, who is putting his professional skills to good use to his hobby:
Hello again, I want to present you what could be the final results of my little side project – AI generated LEGO minifigures. It is a continuation of the first article published here a couple of weeks ago, so I encourage you to read that first if you've not yet done so.
Last time, the algorithm that I have developed generated quite blurry, but recognisable pictures of LEGO minifigures that do not exist. I was not entirely satisfied with the results, so I decided to continue my work.
In the first article I explained that I used an AI model from the family of generative models (GAN). That model, taught using minifigures photos, took 100 random numbers as an input and generated 128x128px minifigure images. This time I used it as a base model that generates input to the next model called CycleGAN.
This type of model is used to transfer styles from one set of data to another, for example to transfer the painting style of a famous artist to a photo taken by you. This time I used it to transfer the style from real minifigure pictures here at Brickset to the pictures generated by the first model. By doing this operation I wanted to enhance the quality and resolution of generated minifigures and the results are very satisfying to me:
I am astonished how well this type of algorithm worked. Some of the simple, classic minifigures almost look like actual photos! In some of them you can also see Brickset logo generated by AI. The advantage is that you can generate these minifigures without any limitation.
Of course these images are selected by me, in many cases the generated pictures are rather abstract:
One common problem with the family of GAN models is a problem called mode collapse. This is the situation where many various inputs lead to one specific solution. You can see it below, where a few various minifigures have almost the same torso print. I will try to solve this issue in the future.
Here are some of my favourite non-existent minifigures that the system has generated.
Here we have some space cowboy with white hat and bandana.
Something that looks like new Batman minifigure.
Some new Stormtroopers.
New Marvel villain?
Some creature from snowy planet.
I am very satisfied with the results. When I was starting the project I never thought that the software models could do such a good job!
Who knows, maybe this is Series 165 of Collectable Minifigures? ;)
86 likes
26 comments on this article
Better than the current state of printing on some Lego figures.
Marbled moulding of minifigure parts could be interesting.
Looks a lot like some customs and knock off minis. Good, but not as sharp as we'd expect.
Thank you for sharing this very interesting article.
In the second set of images it looks like those minifigures have been stuck on a hot light bulb.
Feels like it's time for that annual report on the state of non-LEGO products as these give off major LEPIN vibes.
These are fascinating! Excellent work, Pawel! Great to see people merge passions into projects like this.
They are better now. But some of them scare me a little bit.
The improvements since the last article are incredible!
So interesting!!!
@CT8088 said:
"Thank you for sharing this very interesting article.
In the second set of images it looks like those minifigures have been stuck on a hot light bulb."
IMO they all look melted, to some degree.
3rd image looks like a new Alpha Team.
I think the best one so far is the one in full red from the cover photo. It actually looks like a minifig that has seen years of play, possibly including some time spent in a sandbox. A lot of the others still have a halo effect, where light parts have a very dark outline to define them (the fedora is a particularly bad example of this). Also, every minifig you included in the example of having the same torso print also appear to have the same leg print, so they're nearly identical from the neck down (as are the solid black minifig in the cover photo, and your space cowboy). However, the same torso print _does_ pop up without the matching legs on other minifigs within this article (two with brown legs and one with green legs in the first batch of eight, Batman, and the very last two minifigs in the article). In some cases, it becomes possible to pick out individual prints that some of these are based off. Just in terms of heads, brown pants in the cover photo has a Jake Lloyd head from Ninjago, blue-hair looks like one of the heads from the non-NBA basketball theme, and the first one with a fleshie head looks like an early Han Solo print. Of course, in the last group, a classic smilie rears its ugly head.
@Rimefang:
Looks more like professional replicas of many custom minifigs. There's not a single instance where you can identify a badly cut piece of printer paper that's glued onto the torso, but there are definitely some that look like someone copied that look and either used waterslide decals or printed it directly on the torso. And for as messy as they still are, they look more crisp than your average Sharpie deco (excluding those of us who very neatly fix the corpselike pallor of a fleshie head with the neat application of a layer of Sharpie Yellow).
@Jelippo said:
"Better than the current state of printing on some Lego figures. "
Hahahahahahaha! Easily the best comment of the year!!!!!!!
I will have nightmares after seeing the 2nd group, thanks.
In the words of Hayao Miyazaki, "I strongly feel that this is an insult to life itself."
I’m joking though, I always find AI results pretty interesting.
These are my new sleep paralysis demons
Now we know what the creatures will look like that will take over the world in 10 years. Thanks!
Very interesting! Thank you for doing and posting this :)
Horrifying
Mega Bloks is ready to hire you today.
Erm... I have to admit, some of these minifigs are REALLY creepy...
So... Where is the Annual Report on Communist LEGO, I wonder?!
Hey Pawel, would it be possible to get those as art prints (or at least hi res pics for printing them at home)?
@Lari said:
"Hey Pawel, would it be possible to get those as art prints (or at least hi res pics for printing them at home)?"
Hi, these are unfortunately only 256 x 256 pixels.
@pablo94 said:
" @Lari said:
"Hey Pawel, would it be possible to get those as art prints (or at least hi res pics for printing them at home)?"
Hi, these are unfortunately only 256 x 256 pixels. "
I think there is another kind of AI that will scale up small images into HD, don't know how well it would work in this case though
Unfortunately the embiggening of the pictures mostly doesn't work for me. The stay the same size but just 'pop out' a bit. Pity because I'd like to watch tham in a bit bore detail.
I use Firefox 79.0 on MacOS 10.14.6
@tsa:
It's an issue with the size of the pictures. Whatever the size of the uploaded picture is, that's the biggest you can zoom in. Sometimes you can zoom in twice, once by clicking on the image in the page, and a second time by clicking on the arrows in the lower right corner of the pop-up pane. Sometimes the image was small enough that the full-size version is what you see embedded in the article. This just happens to be one of those times.