Reimagining LEGO sets

I imagine this will be my ignorance of AI, but what's the wider benefit in creating an image based on an image? I could understand feeding in all sets from a (sub)theme resulting in ideas for sets that mesh with the established products, but like for like seems a bit moot? We have an image of an object, and AI creates, to varying degrees, a less refined image of it?

I think in terms of A.I. it is certainly interesting as a technical demonstration, and if you squint at a thumbnail it does look impressive... but blowing up the image to look at the details reveals the same issues as all A.I. images seem to have, poor attention to detail, muddy lines, and a general sense of LSD induced madness. Unless there is a significant leap in A.I. image tech, it seems those problems will persist; although it might be seen as a useful tool by more serious human artists at least in the pre-visualization and idea generation phase.

Where I have seen more use is in the text-based A.I. generators. While A.I. writing is often soulless and bland, and struggles with nuance in historical subjects... it is extremely great at offering quick and clean summaries of well established STEM subjects. Ask say Chat GPT to "explain the Fourier Series in terms a seven year old child can understand" and its a surprisingly simple summary that can then be a bouncing pad into more complicated study in traditional textbooks. For all the concerns of plagiarism in essay writing, I can see A.I. becoming useful in a classroom environment as a study aid for those who learn how to ask it useful prompts.

But back to art, I would love to see some highlighting of human LEGO artists. Even LEGO's own official work (the Art Book for the short lived Bionicle G2 comes to mind) is full of amazing art that is worth drawing attention to.

I must say, as an author, I am a little surprised how the sentiment has changed over the last 1.5 years since I posted my last article. However, some of you are right, the interest rate in AI has exploded over the last few months and you may feel overwhelmed by it. But I think it is just a wonderful tool that may help artists in work. I do not call myself an artist by no means. :)

@bulbathor said:
"I imagine this will be my ignorance of AI, but what's the wider benefit in creating an image based on an image? I could understand feeding in all sets from a (sub)theme resulting in ideas for sets that mesh with the established products, but like for like seems a bit moot? We have an image of an object, and AI creates, to varying degrees, a less refined image of it?"
In this case, I think it may be cool as an app for children in which they upload a photo of their MOC and generate artwork of it. :)

@Huw said:
"People don't come here to read negative diatribes so, please, if you have nothing constructive to say, don't bother commenting.

If you have prepared articles featuring real artwork then I will be happy to publish them.

Thank you.

"

Thank you! I’m was surprised to see so many negative comments in response to this article and the previous one (minifig AI part 3). Not to judge any particular one of those comments, but looking at them in the aggregate, it’s hard not to feel like there’s an underlying fear of AI. I think that’s understandable (and I feel it myself), but a little more self-awareness before clicking the “Post comment” button would be nice.

These are pretty cool! I wouldn’t mind a whole article on just the modular buildings and what they might look like

That is one funky-looking Optimus Prime. And parts of the Galaxy Explorer look like they're melting. That said, the Pickup Truck, Parisian Restaurant, and Gingerbread House look pretty cool. I particularly like the way it made the House look like it has runny frosting. I'm amused by what it did with the Mandalorian Helmet's nameplate.

I agree with moderators deleting comments that break the community guidelines, but deleting comments because they have a negative sentiment or disagree with the author is not what I expected.

@mrzeon said:
"I agree with moderators deleting comments that break the community guidelines, but deleting comments because they have a negative sentiment or disagree with the author is not what I expected."

Agreed. Given the way that AI is being misused at this early stage, often at the expense of peoples’ livelihoods, it’s deeply troubling and disappointing.

Yes, it’s exciting to see what AI can do, but it is hurting communities of artists and not allowing people to recognise that because it’s “negative” is a stance I didn’t expect this site to take.

Interesting. You would think this should be pretty easy for AI.

In some instances (particularly the safari truck) the AI has recognised it was a vehicle and basically just produced an image of a vehicle, even if some of the details were completely different.

In other instances (Parisian Restaurant) it failed to appreciate that buildings are typically constructed using solid materials and straight lines, and instead produced a fuzzy mess, even when it would have been more accurate to literally just reproduce what was inputted.

@mrzeon said:
"I agree with moderators deleting comments that break the community guidelines, but deleting comments because they have a negative sentiment or disagree with the author is not what I expected."
That comment makes this thread more understandable. Huw’s comment on negative responses was after only three comments which made no sense. However, if many have been deleted then it does. I am similarly concerned comments were deleted just for being negative. Only allowing praise does not show a balanced view.

@tmtomh said:
"it’s hard not to feel like there’s an underlying fear of AI."
I don't think it has anything to do with fear. Just that it's something that a lot of people don't like the look of (although a lot do).
Much the same as autotune or botox. More stunned than stunning.

I would like to make it clear that I'm sure the author was acting in good faith with their article, but AI has become such a far-reaching issue across multiple industries and people are right to be negative. For every person seeing AI as a fun tool to create imagery they couldn't previously, ten other people are looking for ways to capitalise on the technology for a profit at the expense of already vulnerable industries or communities.

Voice over artists are being replaced by AI-generated vocal models, and in some cases their own voices are being used to train the AI without their consent.

Artists both amateur and professional are having their work used (once again, without their consent) to train AI generators like Stable Diffusion or DeviantArt's built-in AI "DreamUp".

American sci-fi and fantasy magazine Clarkesworld had to close submissions in February after being overwhelmed by AI-generated written submissions (over 5 times their usual number of submissions). Read more here: https://twitter.com/clarkesworld/status/1625982159856041985

Again, I understand that to many it's an exciting new tool to create imagery with, but it's too closely attached to a number of issues likely facing a lot of people who read Brickset in a very head-on way. I think any discussion of it should take the way it's also being abused into account, and more consideration should be taken as to whether it's worth posting about it at all.

A potentially good, useful, fun tool is being ruined by race-to-the-bottom capitalism, and is taking peoples' livelihoods with it.

Allow me to restate my comment in a constructive form: It is actively harmful to artists whose stolen art forms the backbone of generative AI models when generative AI articles are posted in forums that have as much reach as this (truly fantastic) website without explaining that the generated art is at its core based on the unlicensed use of other artists’ work. It also makes people (me included) squirm to see sites driving engagement off of something that is almost certainly legally copyright violations (at least in the US, I do not claim to be familiar with other countries’ laws on artists rights), and certainly feels immoral. I would normally just drive by and not care, but I find the frequency with which people do not realize all generative models are built on a backbone of theft concerning, and I think it is important to contextualize this, or risk being complacent in it.

@Huw said:
"Thank you @Jack_Rizzo for your insightful comment."

No problem, and apologies as my initial comment was a bit terse. I do appreciate you giving me a chance to discuss my concerns in more depth.

Feeding stock photos into a machine-learning dataset without permission from the copyright holders seems like a bad idea to me.

Additionally, the results simply aren't interesting. It smoothes everything out and take away the charm of what LEGO sets are while revealing that the system can't interpret the shapes and colors correctly. A waste of time and energy that really has nothing to do with LEGO news and should not be featured here, IMO.

Proliferation of something that devalues what artists do isn't something I can support. Being angry and negative about something that threatens my livelihood as an artist is a valid response. Deleting any comments really isn't in the spirit of free speech.
Please keep AI out of LEGO, is that better?

Hmm, I think I might have been more…impressed might be the word, interested maybe, if the A.I. hadn’t had the original image to go on, just the description. Making an uncanny valley version of an image it’s been handed doesn’t seem as cool as what it might have come up with without such a prompt

Very interesting article. I was quite impressed by the AI's rendering of the red pickup truck, but after seeing the other images the limitations of the technology become obvious. The X-wing for example is straight-up hilarious!

A question for the artists on here: What would you say is the difference between AI creating art based on existing art it has been trained on versus a human artist creating art after having been exposed to other people's art?

@Huw said:
"People don't come here to read negative diatribes so, please, if you have nothing consructive to say, don't bother commenting.

If you have prepared articles featuring real artwork then I will be happy to publish them.

Thank you.

"

I've seen far less constructive comments on plenty of other brickset articles (the minecraft bigfig reviews in particular come to mind), why does this only apply when AI art is involved?

@Jack_Rizzo said:
" @mrzeon said:
"I agree with moderators deleting comments that break the community guidelines, but deleting comments because they have a negative sentiment or disagree with the author is not what I expected."

Agreed. Given the way that AI is being misused at this early stage, often at the expense of peoples’ livelihoods, it’s deeply troubling and disappointing.

Yes, it’s exciting to see what AI can do, but it is hurting communities of artists and not allowing people to recognise that because it’s “negative” is a stance I didn’t expect this site to take."

Thank you for this.
I actually am a professional artist and I appreciate that.

What I've learned from this is that AI generated images look like they're very detailed models made from cake...

As an artist, I share the absolute frustration (a reaction to injustice) at the unethical practices used by the AI art generators, and I believe it should be called out. It's immoral and seemingly designed to do away with paying people what they're worth, so I can't support it. People can't appropriate land and just call it theirs, can't breach copyright, and to my mind this is what's happening in the setting of artistic works and people's livelihoods.

However, this article is instructive as it shows that AI art generally can't match the design of the original Lego sets - of human input, ingenuity, creativity and charm. There is a purpose and sense of storytelling to the best Lego builds, and this piece highlights what AI art can't do.

How about an AI that can take images of real-life objects and turn them into promo shots of fake Lego sets?

@xboxtravis7992:
As mentioned in previous articles, similar systems for creating human portraits have been running so long that they have become quite hard to distinguish, needing regular input to tell it which ones we find realistic and which look fake. These are much earlier in their progression, so they don’t have that benefit. I suspect another major difference is the human and other similar AI projects have been able to crowdsource the learning phase by getting people all over the world to stop by their sites and respond to generated images, where the author may be handling that process solo.

@NotTheLastBrick:
I was thinking something similar regarding the nose of the Galaxy Explorer. It clearly understands the shapes involved, but not the purpose of those shapes. It has twin pokey things in place of what are clearly guns, to anyone who isn’t a LEGO exec. And the engines being square may fit with popular art, but if you’ve ever seen a slo-mo video of a rocket engine firing, square is physics’ plaything.

Conversely, the truck and the duck look very believable. Optimus Prime looks like it came from their R.E.D. line. Manmade shapes seem fairly simple to understand, where it’s the natural stuff that stumps it. Looks like it just gave up in a fit of rage and scribbled the tree in before it even got to adding the woman.

@tedgarb:
Multiple times, the US court system has stressed that images posted on the internet are free to view. As for the learning part, if you judge it by human standards, new artists learn their craft by studying the works of those who came before them. They then go on to compete with those they learned from (at least those who are still active). Nobody’s protesting against that.

I get where the perceived threat is, since this isn’t about a machine being taught to produce art for itself, but so any hack can plug in a description and have a machine crank out art they can claim authorship of. This gets tricky when you consider Naruto v. David Slater et al., where PETA tried to claim copyright of a photo on behalf of a macaque who allegedly took a selfie (it’s unclear if the monkey that took the photo is the one in the photo), while Slater claimed he held the copyright. Courts ruled that he didn’t take the photo, and the monkey can’t hold copyright, so the image is in the public domain. The same probably holds true for AI-generated works of art.

@Ridgeheart:
My understanding of “flat-out plagiarism” is the copying and republishing of whole or partial works, which this does not appear to be. At present, there doesn’t appear to be any law governing this, which is not to say there won’t or shouldn’t be any in the future. As I said above, presently the only legal status these works appear to have is that they can’t be copyrighted because they’re not created by a human.

@ChocolateCrisps:
There also appears to be a glass bottle inside that Gingerbread chimney, so maybe be careful when you’re chowing down.

It is quite impressive to see what these AI systems can come up with, but I think it is important to realize there's very little true intelligence at work, it's just a system quite good at mixing things up that it was fed with. Everything it makes is inherently derivative, it cannot create anything actually new. In that regard true AI is still far away.

As for these images, some are better than others, but they also clearly show the limitations of the system. In the end it's just a dumb system mixing things up it doesn't understand.

It would be more interesting I think to make an AI Legofy non-Lego objects, and see if it comes up with things that could actually be build. Somehow I very much doubt that though.

@PurpleDave said:
" @tedgarb:
Multiple times, the US court system has stressed that images posted on the internet are free to view. As for the learning part, if you judge it by human standards, new artists learn their craft by studying the works of those who came before them. They then go on to compete with those they learned from (at least those who are still active). Nobody’s protesting against that."
Free to view =/= free to use

If I see a picture on the Internet that has copyright on it, there are laws and regulations that limit what I can do with it besides looking at it. In many cases, I cannot use it for my own (commercial) purposes without consent of the holder of the copyright.

Also: AI systems don’t truly learn. They are not creative. They cannot create something original. They don’t understand what they “see” or what they produce. They are not artists.
They are a collection of algorithms that process data that has been fed to them, which they cut up into pieces, match it to keywords (of which they don’t truly understand what they mean) and present those pieces in a different order. It’s an impressive piece of technology, but what it produces is not art.

@Tc99m:
You’re talking about laws that govern the use of whole works of art, or at least recognizable portions of them. In this case, you can’t look at the output and point at any one part of it as being plagiarized. To do so, you’d have to be able to first identify what other work was plagiarized, which you can’t really do. It’s easier by far to look at something produced by a human and pick out what other artists or works of art they were inspired by because we are selective in what we like and dislike. These programs learn from everything they come across, so their output is derivative of both everything and nothing.

@pablo94:
As these programs become more prevalent, and especially as their output is commercialized so it competes with real human artists, I fully expect laws will be passed to govern the use of their output. I don’t know what those laws will be, but I expect they will be more protective of human output than this. For now, though, they don’t exist, beyond that my understanding is this output can’t be copyrighted by anyone because no person created it.

@Ridgeheart said:
" @M_Jibril said:
"Very interesting article. I was quite impressed by the AI's rendering of the red pickup truck, but after seeing the other images the limitations of the technology become obvious. The X-wing for example is straight-up hilarious!

A question for the artists on here: What would you say is the difference between AI creating art based on existing art it has been trained on versus a human artist creating art after having been exposed to other people's art?"

There's a difference between "learning from", "being inspired by", "paying homage to" and "flat-out plagiarism". I understand your analogy and I appreciate your desire to learn more, but this isn't a budding artist who develops their own style by studying those who came before them; this is Victor Frankenstein, cutting and pasting limbs together without permission - and then, selling the end-result.

And I for one, would have liked the option to opt out of having my personal tomb desecrated."

Fair enough, thanks for taking the time to respond.

@thor96 said:
"I must say I have enough of the complaints from "artists" that the AI model "steals their artworks without their consent". This is not the way AI models currently work. All of those artists do exactly the same a the AI models- they learn from other artists and nature and consciously or unconsciously incorporate things they like into their own style. No man is a lonely island, no artist creates anything that is truly original. It's always an amalgamation of the past and present things and ideas that are already there. Those artists who understand that will adapt, those who think they are the only originals will be forgotten, but it will not be the fault of AI, it's just the nature of life- everybody cannot be successful. Now, about the article- I really like the parisian restaurant and safari car, they look eyecatching! "

I must say, I've had enough of people with no training in the arts deciding whether or not I have a right to be furious about potentially losing my livelihood and devaluing what I do for a job.

As someone said earlier in the comments, the I in AI is a bit of an overreach. No real intelligence, just a lot of data and lines drawn between those datapoints. Impressive, but not intelligent.
A clear example has a lot to do with Lego (I use it all the time when talking to people about 'AI'): try to create a Lego set out of a real life image. AI doesn't understand the rules behind Lego, so the results are crap. Maybe if Lego released all the measurements, logic and rules of all parts and feeds it to an IA, then maybe, but it won't be able to figure it out on its own.

@thor96 said:
" @BernardBlack said:
" I must say, I've had enough of people with no training in the arts deciding whether or not I have a right to be furious about potentially losing my livelihood and devaluing what I do for a job."

That's quite funny, since I am myself an artist. But I try to keep my ego in check, that really helps in life."

I'm really glad you have your ego in check. Well done you.