Piqabrick: real-world tests

Posted by ,
View image at flickr

Piqabrick is a project on Kickstarter that aims to deliver a system that will identify parts and minifigures automatically.

As I mentioned on Tuesday, I met the project team, Stefano and Claudio, in Italy at the weekend, who gave me their prototype unit to play with. I've now had a chance to put it through its paces and perform some real world tests.

The results surprised me...


The main physical deliverable of the project is a camera and lighting unit. This one is a pre-production model, so has been 3D printed and is not constructed the same way that the final version will be.

View image at flickr

It's exactly 16x16 studs in size and has to be mounted on top of a 12 brick high box in order for the camera to be the correct distance from the base and subject. You can build the box however you want, and in whatever colour you want. I went for a nice simple design which isn't the sturdiest but is perfectly adequate.

The white card at the bottom is provided with the kit and has very fine lines on it which helps the camera auto-focus.

View image at flickr

The camera is operated through the Piqabrick website, which at the moment is a work in progress. It currently provides just enough facilities to take photos of parts to add to the database, and to scan parts to test the image recognition. I won't therefore post too many screenshots or pass judgment on its utility yet.

Around 300 parts and minifigs have been scanned into the system so far, which is enough for test purposes but is of course a tiny fraction of the 70,000 in the BrickLink catalogue that need to be done if the system is to be useful.

Project creator Stefano told me that at the moment it works best with compex parts rather than simple bricks and plates, so I thought I'd add a couple of Technic pieces and then see if they would be subsequently recognised.

Adding parts to the system requires taking multiple pictures of them from various angles. The more pictures, the better: I think I took a dozen or so. Of course, you would not need to do this normally because when the project launches the image database should be fully populated.

After a few minutes, I switched from 'add part' to 'scan part and identify it' and it did so without any problems.

View image at flickr

View image at flickr

These parts are very distinctive so that perhaps wasn't much of a test. I therefore tried with two Alien Conquest soldiers that are identical except for their heads, which, to make it harder, are under helmets.

They weren't already in the Piqabrick database, so I had to find out their BrickLink identifiers and then add them, again, by taking multiple pictures from various angles.

Once done, I asked Piqabrick to identify them. The results were spot-on:

View image at flickr

View image at flickr

What's more impressive it that it even managed to identify one with the helmet removed, despite me not adding pictures of it in that state to the Piqabrick database.

View image at flickr

Of course I have not tested it exhaustively yet but I believe I can conclude that the image recognition works effectively and perhaps better than I expected.

The system is entirely reliant on an accurate library of images of every part and minfigs to be identified and it's going to be some undertaking to provide them for all 70,000 of them by the time the product ships in February. It was not difficult for me to add four for test purposes, but it probably took 1 minute per part which didn't include identifying them in the first place.

Claudio said, in the BrickLink forum, that they aim to have 90% of the parts and figs added by launch and I have no reason to doubt that they can do it, given enough manpower, but it's going to be a huge job!

There has been a lot of discussion about the usefulness of Piqabrick, in comments to previous articles and in various forums, and in particular questioning who's it aimed at. It's probably too expensive for the casual user but having conducted these tests today I think that those who need to identify minifigures and printed parts accurately are likely to find it particularly useful, as are BrickLink sellers who want a quicker way of determining the identity of unfamiliar parts without squinting at tiny writing on them or wading through the BrickLink catalogue.

The project is now about 70% backed with 9 days to go. Pledge your support today to help make it a reality. Who knows, if this is funded, it could be the first step towards an automated parts identifying and sorting system...

If you want me to try some other tests, suggest them in the comments. If I have the parts needed I'll do so.

67 comments on this article

Gravatar
By in United States,

Sorry if I missed this in the article, but do we know what the retail price for this will be?

Gravatar
By in United Kingdom,

So basically, if a parts not scanned in, it can't identify it... That kinda limits its usefulness. Not to mention you admit in the review "its not good with simple shapes yet", which kinda says it doesn't really work as intended to my mind if it can only work with complex parts and minifigures etc.

Also, I'd ask again... Why wouldn't a smartphone app work, if given the same 12 brick high and 'finely lined baseboard' (which is pretty obviously a sizing guide framework) parameters?

Gravatar
By in United Kingdom,

€145 but there are much cheaper 'early bird' rewards available currently.

https://kck.st/2PgO51X

Gravatar
By in Netherlands,

For testing I suggest to use the list of parts that Quertyboy (a well known Canadian seller on Bricklink) made for BL sellers and buyers. The list is on his own website: http://www.qwertyboy.org/files/parts.php
The parts listed there are just a small sample of parts that aren't easy to look up in the BL catalogue and many sellers/buyers are struggling with it.

Gravatar
By in United States,

Does the color detection work in their software yet? Would be interesting to see how it does in identifying some of the older colors or shades of current colors.

Gravatar
By in United States,

How does it work with transparent items, like tiles, bricks, and (to appease the huwbot) clikits?

I love this review. Thank you!

Gravatar
By in United Kingdom,

@GrizBe , "if a parts not scanned in, it can't identify it". Yes that is true, but the team has pledged to have 90% of the BL catalogue scanned at launch so it should not be an issue.

"its not good with simple shapes yet" I am told that not all the available image recognition algorithms have been added yet but they will be by launch.

"obviously a sizing guide framework" it's possible, but I was told it was to aid auto-focusing and I am also told that the final deliverable will have a fixed-focus lens to avoid hunting and the delay that entails. So, perhaps these lines won't be necessary then.

@gpfleging , I will test various shades and let you know but I have no reason to think that, if parts in old colours have been scanned, it won't recognise them.

@dutchlegofan50 , once they've been added they will be recognised.

@alyxavior, I'll try!

Gravatar
By in Netherlands,

@Huw "€145 but there are much cheaper 'early bird' rewards available currently."
It depends on the country. I just pledged for the Early bird > Piqabrick White Top which is € 111. With shipping it was € 125.

Gravatar
By in United Kingdom,

Great review, @Huw.

How are they sourcing the 70,000 parts to scan them in? Is there an existing database they can repurpose? Or will they be drawing on support from their users?

By the way, you wrote ‘Stafano’ near the beginning rather than ‘Stefano’.

Gravatar
By in United States,

I think this is a great idea, and can only get better overtime.

I have pledged, I hope they reach their goal!

Gravatar
By in United States,

This seems awesome, a lot of people in AFOLS of LEGO will find this useful.

Gravatar
By in United States,

Also, is the bottom card black on the other side for lighter colored pieces?

Gravatar
By in United States,

I'm also curious how it treats color. For instance, can it tell an old, yellowed white plate from a beige one? Does it sense ambient light temperature? I assumed the solid, 3-sided black box from the previous articles was a necessity for lighting reasons, so it was interesting to see your open-sided white box.

Gravatar
By in United States,

Would really appreciate seeing a test on tires and wheels, Huw. Having new eye problems (with age) ?? it’s gotten to be a bummer to wait for the youngest to get home to verify. PS for this reason, I’ve already become a backer, but would love to see these items tested.

Gravatar
By in United States,

Oh, and about size of items, what about Duplo figs?

Gravatar
By in Slovenia,

I am sceptic about database. They need physical parts to scan. How will they obtain all 70.000 or even 90% of those parts? Some are really rare or expensive or even not a sigle one on BL. And most valuable usage will be to identify rare odd parts. I don't need device to identify 2x4 plate, but I struggled with part 4503176 Vehicle, Brush Axle 1 x 8.
I wish them all good of course!

Gravatar
By in United Kingdom,

Thank you for taking the time to test this for us, Huw.

Even better, thanks for giving ADU some love. Alien Conquest forever!!

Gravatar
By in United Kingdom,

@Subix, I do wonder that myself to be honest but I'm prepared to give them the benefit of the doubt at the moment.

Gravatar
By in United States,

Thanks for the review, Huw. I signed up early and really hope they meet the funding goal to get this into production. If they do manage to get 90% of parts into their database, I'm sure folks in the forum can identify the other 10%, and those will get added to the database as users identify them.

Gravatar
By in United Kingdom,

Yes, there will almost certainly be an element of crowdsourcing involved, particularly as new parts are produced.

Gravatar
By in United Kingdom,

I wish I had this already.... God knows how much time I've wasted recently trying to I.D. a bunch of random parts received as part of an incomplete set.

I agree with some of the reservations expressed in the comments, but I'm reassured that the fundamentals seem sound. The database will eventually catch up (after all - Bricklink, Brickset etc. all had to start somewhere....) and I think it'll ultimately take the frustration out of identifying printed parts and unfamiliar elements so I'm willing to stomach a slow start to help get this product up and running.

Pledged.

Gravatar
By in Canada,

I think it is interesting and could be useful. Looking at the images I see the Lego ElementID, but I think what really would make this useful would be if the database also included:
1) alternate ElementID(s) (sometime elements are given a new ID if it hasn't been used in a while;
2) patternID, elements that come translucent or metallic are given a pattern ID (differs from DesignID);
3) alternate DesignID (Lego has updated a part with a new mold);

I know this functionality would be useful when I want to go to the pick a brick at the Lego store and if I can't get piece I need I can quickly get a list of other IDs I could use.

Gravatar
By in United States,

Is a subscription required for it to work?

Gravatar
By in United Kingdom,

No subscription required.

^^ I'm sure all that can be added over time.

Gravatar
By in United States,

Reading the pledge levels, it does allude to a 29 Euro / year subscription. Many pledge levels are lifetime free subscription, though. Likely some added value in pledging with a lifetime subscription!!!

Gravatar
By in United States,

I believe that the subscription was removed and was mentioned in the comments of the initial Piqabrick brickset.com article.

Gravatar
By in Israel,

@Huw are you able to upload images or does it have to be one taken by the device itself?

Have you had a chance to ask them about API access?

Gravatar
By in United Kingdom,

What does it do if, for example, you try to get it to ID one of those figures turned upside down, so that they're indistinguishable from each other? Does it give you a list of possible figures?

Gravatar
By in United States,

This is a really cool thing. What's the cost again?

Gravatar
By in United Kingdom,

^^ Yes I believe it will, and give a % likelihood of a match when it launches, but it doesn't at the moment.

Gravatar
By in United Kingdom,

@Huw, an interesting test would be to scan pieces similar to (but not the same as) the ones you taught it and then see if it misclassifies them as the ones it knows about. If so then that's much less impressive - after all, it's considerably more likely to get it right if it has only one or two similar parts to consider, if it is just looking for the closest match at the moment.

Gravatar
By in Sweden,

I really hope this works as advertised and gets funded! I want mine to go through the huge pile of minifigures I have!

Gravatar
By in United States,

Pledged hope they make it!

Gravatar
By in Belgium,

This is impressive and looks very cool.

Gravatar
By in Germany,

Thanks for your report and tests Huw. I think your articles will help the project reach its goal. Without them, I would have not supported it. Sounds like a cool gadget.

Pledged.

Gravatar
By in Slovenia,

datsunrobbie: I have many old and odd parts and offered to the team to scan alias be beta tester. Not accepted.
In case they do have on stock 90% of parts, 1 minute per part, 8h day =131 man days to scan them all. Quite a team of people to achive that in timeframe they set.

They might offer all users to scan parts, send to their database and SW have option to use only offical database or unveryfied user populated database.

Again I wish them all the best and hope to fullfill this unique gap on market.

Gravatar
By in Netherlands,

I distrust the dependency on an online picture library. What's to stop them from monetizing the library in the future?

Gravatar
By in United Kingdom,

@Hobbythom: Considering they originally did want to charge a subscription, its a possibility they could go back to wanting to do that.

Gravatar
By in Germany,

Still on the fence about it. I would like to have a more open database as a backend. Either Bricklink or one that has a local copy. I do not like to rely on a database at a company elsewhere forever. It is not "my" database and can disappear for whatever reasons.

Gravatar
By in United States,

Looks pretty awesome. I wonder how it would respond to a made-up minifigure (like iron man helmet, darth vader head, random torso and legs). Or a torso that maybe has the incorrect color arms and hands attached.

Gravatar
By in United Kingdom,

^ It will give several possibilities with a % likelihood of a match.

Gravatar
By in Italy,

@Nivst : API interface is not available right now, but it's in our roadmap.

Gravatar
By in United States,

@Hobbythom, @GrizBe, @Mickitat You're right, there's really nothing to stop Piqabrick from monetizing their database, any more than anything is stopping Huw from monetizing the Brickset database, or the owners of Bricklink from doing the same. Sometimes you have to have a little faith that people will do the right thing.

Gravatar
By in United Kingdom,

@datsunrobbie : Fair enough. Thing is though, Brickset was set up with the intention of being free and always being free, and Bricklink makes money from the market. This however was originally set as a subscription model. Theres a big difference in intended mindsets.

Gravatar
By in Netherlands,

The difficulty of classification problems scales with the number of possible outcomes. Currently, there are only 300 parts in the database. I'd be very interested to see how well the system operates if there are 3000 or even 30.000 parts in the database.

Gravatar
By in Netherlands,

@datsunrobie You are right. But once Huw figures out he's sitting in a goldmine I am not stuck with a useless 115$ machine.

Gravatar
By in United States,

Unless it gets made, we'll never know how well it works with a larger data set. I think it is worth my $120 to help see that it gets made.

Assuming the tech is pretty much the same as PiqaPart, I'm very optimistic about it being able to scale to a database containing the majority of Lego parts.

Gravatar
By in United States,

@Huw does it work on the perennial favorites Clikits? Or Galidor, Belleville, & Znap. I assume they're not in the database but I would like to see how it does with those oddities.

Gravatar
By in Australia,

Excellent review, but I’m still thinking of passing.

Gravatar
By in United States,

A useful application would be in inventory management of your collection... cataloging the part numbers with bin numbers, drawer labels, etc... I'm sure they are working on such software to make good use of this cool little invention.

Gravatar
By in United Kingdom,

Fascinating concept. Clearly the design and construction still in it's infancy. Can only imagine that the team behind it fully appreciates that the database itself is the key to rolling out a successful product as the scanner itself is redundant without it. Buona fortuna ragazzi.

Gravatar
By in United States,

I agree with many here that a smartphone app seems like a far better (and cheaper) application. There are several similar app for trading card games. None that I can find for Lego though. Why would I need a camera and a light box when I have countless cameras in the form of phones already. I just need a database. Seems like a no brainer.

Gravatar
By in United Kingdom,

@RevN3 - if you go to the original article on Brickset they talk about why it's not an app (though they will go back to this down the road to see if viable), predominantly no phone cameras are the same between brands and models and this box give best and consistent light and distance.

@GrizBe - the subscription was removed after listening to feedback, all backers that say subscription won't have one. Subscription is gone and won't return, they believe in the product so much they will eat the cost on upkeep a subscription would have covered.

Someone mentioned losing their database. You have the option to export your database to a spreadsheet.

I have backed this project.

Gravatar
By in United States,

A pretty awesome machine. I probably won't get it, at least for a while, but for minifigure collectors this is a godsend!

Gravatar
By in Singapore,

I did the math.

100% Parts = 70,000
90% Parts Promised to be in by shipping (1st Feb) 63,000
Parts per minute = 63000 minutes
63000 mins to hours = 1050
Working hours, 9-5 = 8 hours a day
1050 / 8 hours = 131 days
Days remaining till Feb 1 = 2 months 13 days
Total remaining days from 17 Nov to Feb 1 = 75 days
Number of Staff in GETCOO: 1. Stefano, (CEO), 2. Claudio (CTO), 3. Jona (Visual Design), 4. Roberta (Digital Media Manager)
Possibility of getting it done = for sure
The likeliness of getting it done = not really

(Yes, they can hire 4 interns, and get it done.. but I'm doubting they'd do that considering they need the funding to get this project running... just my 2 cents)

They will essentially need 2 persons working over weekends from now till shipping date to get the promised 63,000 parts in. Very possible to achieve, but highly unlikely. Good thing the part scanning can be crowdfunded.

My only question is to Huw is, you scanning your own part and then identifying it is like taking an exam you already have answers to. Since you've offered in your article, could you test items that you didn't put into the database and verify for accuracy?

I wasn't convinced initially, but Brickset's review certainly gave me more confidence because of such a positive review (or rather, non-negative).

Thanks!!

Gravatar
By in United States,

So how well does it identify large baseplates, the 32x32+ range? (Sorry, someone had to ask that one...)

@datsunrobbie:
If you'll recall, Bricklink already went through that scuffle when they announced new Terms of Service that gave them copyright to everything in their database. They ended up having to get permission from every person who had previously submitted images to keep using them, and in any case where they were pointedly denied permission or could not contact the original submitter, they had to delete the images and replace them with images from submitters who didn't mind sharing copyright with Bricklink. This could end up in the same situation. If people submit images on the understanding that there will never be a subscription fee, and they decide to add one, they may find that the people who originally submitted images start asking to have their hard work removed from the database.

I personally think they'll have to start charging something, but that $29 annually is going to feel excessive to many. If they sell $1000 of these, that would bring in $29,000 _per_year_ unless people drop subscriptions. Currently you can get much of the same service by simply posting images to the Bricklink forums. It may take a few days to work through a large list, but people really do race each other to try to post the answers first. And that service is 100% free, and I've yet to see it fail (in cases where one person posted the wrong answer, it was always corrected by someone else).

Gravatar
By in Germany,

@PurpleDave - you are right of course. There is always a way to identify a part or a minifig. But to be honest, this thing simply looks fun to use. Which taking photos and posting to the forum simply is not. I do expect the database to be initially really "good". It will need to be filled by their users. Let's face it - parts with stickers alone are incredibly super annoying to identify AND put into the database. I am more and more leaning towards backing the project.
Edit - it is also nice to back projects to see if they work out. Kickstarter is not a shop. It is a venture capital site - it might fail, but that is a decision everyone needs to make by him/herself.

Gravatar
By in Germany,

It has jumped the financing hurdle. Looking forward to tinkering with it (moved to the hopefully right side of the fence).

Gravatar
By in United States,

Congratulations to the PiqaBrick team. Looking forward to getting it and finally sorting out my minifigures!

Gravatar
By in United States,

Alternatively, you could you know, just LOOK at the brick or figure and figure out what it is that way. Seriously, this thing just seems to be solving a non-existent problem. I honestly don't get what the point of this thing is; it doesn't do anything I can't do with my own eyes, and I can do that for free.

Gravatar
By in United States,

@Troodon My 54-year-old-eyes aren't so good anymore, and it can take a while to find a match when looking for minifigures, even with resources like goatleg to narrow the search. If this works as well as I'm hoping it will, it will identify most figures in seconds instead of minutes. With over 1000 minifigures to identify, I feel it is money well spent. Then onward to sorting through a LOT of loose parts.

If you have good eyesight, plenty of time, and not a lot of parts, then this may not benefit you. I buy mostly used lots from the thrift stores, often with a dozen or more sets of different genres jumbled together. I expect this to speed up sorting quite a bit.

Gravatar
By in Italy,

Piqabrick is now 120% funded as it reached the 55k goal a few days ago. But there are cool stretch goals left to reach!
Hurry up! There are just 30 hours left to get behind Piqabrick and help bringing it to life taking advantage of a 16% off on the retail price!

http://kck.st/2PgO51X

Gravatar
By in New Zealand,

is it too late to pledge for 2 of theses boxes still? after 2 black 1s

Gravatar
By in United States,

@datsunrobbie:
That's certainly valid, especially for people who buy up used collections to part out and resell, where they have no idea what they'll end up finding and need to sort stuff as quickly as possible. I actually like scouring the Bricklink catalog, as the more time I spend doing so, the better working knowledge I have of what's in there. As someone who focuses more on smaller builds, part knowledge is crucial. And yes, I've even had to dig deep into the minifig parts categories on more than a few occasions.

Return to home page »