Brickset search enhancements

Posted by ,

Last week I asked you to suggest ways in which we can improve Brickset. There has been some great responses so far and if you've not submitted your ideas yet there's still time to do so on this form.

One thing that many respondents wanted, including my good friend Bricks For Brains, was for search to be improved. So, I've done a few things which I believe have done so:

  • More results are shown in the drop-down suggestion list
  • The list now shows images and clearer set/minifig/part details
  • Dashes and so on are ignored, so searching for 'X-Wing' and 'X Wing' will all show the same results. However, misspellings, typos etc. are not yet dealt with.

The autocomplete drop-down list will start to show matches when you've entered three characters, at which point it'll show only the first 50. That's because It's possible that there are thousands of matches, and I want to avoid overloading things and burning your bandwidth downloading them all.

As you enter more characters and narrow the search, the maximum number of entries will increase, so you should find what you're looking for by scrolling down the list.

Anyway, give it a go and let me know what you think, whether you notice any problems, or whether you have suggestions for further improvement of the search function.

 

Sponsored content

34 comments on this article

Gravatar
By in Germany,

Nice!

I just tested it with the beginning of "Haunted House" (it's Halloween, after all^^) and wondered why I got signals and garages after the first 3 letters first, then some (expected) "haulers". Then I noticed it's because they have "with Automatic..." in their name and spaces are ignored... Would it be possible to move the hits with ignored spaces below the others? Then again, such confusing combinations probably won't happen that often, and in this case are solved with the 4th letter anyway.

Gravatar
By in Germany,

Great feature!

Gravatar
By in Australia,

Awesome!! Thank you Brickset for always striving to improve :)

Gravatar
By in United Kingdom,

Improved search is always appreciated, thank you :-)

I'm curious about what to expect though. Let's say I'm looking for Sandcrawler sets.

After typing "sand" the first few results subtheme categories followed by what seems like unusual matches [047-1, 181-1, 214-2, 214-3]

Adding an extra character, "sandc" and the subthemes disappear but the results are quite puzzling [903-1, 3343-1, 4117-1, 4679-2, 5042-1, 5052-1, 5053-1, 5065-1]. The last is two-way plugs and cable.

Once you get up to 6 characters, "sandcr" then all the surprising results drop away and the four results for sets are the sandcrawler sets.

Should we be seeing the Sandcrawler sets earlier before we reach six characters or is it that to allow as deep searching as possible it means the shorter search terms will see other stuff bubble to the top?

cimddwc's post clarifies things now. That's what I get for typing so slowly. All the results have "...s and..." somewhere in the title.

Gravatar
By in United States,

Thank you Huw!!
Autocomplete isn't a fuss for me, however, being able to distinguish between categories is very helpful! :-) The small image pop-up/drop-down that appears is brilliant!

Brickset is the only way I can find stuff in my own collection (especially when still sorting sets) :-D

Gravatar
By in United Kingdom,

^^ It's interesting that you first searched for something that has exposed some shortcomings!

'sand' will indeed find matches of words ending in s followed by and, and 'sandc' will match '...s and c...', of which there are quite a few, I see!

I guess it's pros and cons whether to ignore spaces or not. I suspect the cons outweigh the pros, though, and I welcome others' views. It's easy enough to include them again if needed.

Gravatar
By in United Kingdom,

I’m really happy with this improvement as it now makes it easier to search for a set, part, minifigure etc

Gravatar
By in Germany,

Very, very good improvement !!!!!!!

Gravatar
By in France,

^^Huw, isn't it possible to show he closest match first? so if you type Sandc, then Sandcrawler is first up?

Gravatar
By in United States,

This new search feature is great!
One thing I don't like though is that it lists all set results before all other results. I just tried typing "plate" and scrolled through all of the suggestions. Even though there were about 150 suggestions, not one of them was a plate.

Gravatar
By in United Kingdom,

^^ Matches aren't ranked. They probably should be, but that's a whole different problem to solve.

Machine learning might be easier: perhaps over time the system could show the results that people most selected when searching for 'sandc' at the top.

Gravatar
By in Italy,

The images in the drop-down suggestion list is a great improvement, often I had to try various suggestions before finding what I was looking for. Thanks!
Maybe parts number could be useful too.

What about improvements in multiple search? (yes, it is one of my submissions! ^____^ ).
Multiple search for parts and minifigures would be great for me. And a dedicated button (in order to avoid fancy formatting) too.

Gravatar
By in Germany,

Awesome! This makes searching so much easier!

Gravatar
By in Venezuela,

Hi Huw, I think that another improvement to the search bar is that you could highlight in a different color sets owned in the drop-down suggestion list

Gravatar
By in France,

Thank you and good job with the search improvements! It's something I did not even think to ask when I wondered what could be suggested to improve Brickset site, but that "bothers" me each time I use it (at least once a day). It was too often tricky to know and write the exact name or determination of an item without errors, but to search the set number elsewhere for instance, and then come back to Brickset, more over when you search for a set that as so many renditions and/or tags as an X-wing - to keep with this example. And now even having the pictures for each result is such a great thing! So thank you to those who have thought to propose it, and to those who have solved the problem.

Gravatar
By in Canada,

Great improvements as always. Thanks for the hard work Brickset Team!

Loving the search enhancements! The added thumbnails are golden.

Is there a way to search for an exact phrase instead of using smart search? For example "sandc" would ignore ...s and c... Results

Gravatar
By in Netherlands,

oh that's great!

Gravatar
By in United States,

This is great, thank you!

Gravatar
By in Indonesia,

"the search now shows images and clearer set details"
WHY HAS NO ONE THOUGHT OF IT BEFORE.

Gravatar
By in United States,

Having thumbnails is awesome.

Gravatar
By in United States,

That was a great idea!

I'm glad I thought of it. ;)
JK when the form came out my mind was a blank

Gravatar
By in United States,

Oooh, nice! I didn't even think of this as something that needed fixing, but it certainly has benefited from the upgrade. Great job, thanks! ^^

Gravatar
By in United States,

Awesome! This makes it so much easier to find a specific version of, say, the Millennium Falcon, which has had numerous sets based off it that nearly all have the identical name. Thanks for adding this!

Gravatar
By in Germany,

Very nice!

The images are a godsend for exactly the reason goldenguy800 already mentioned, it's much easier to pick out the right result from similar names, e.g. System-scale and Microfighter versions of Star Wars ships. A very welcome change that will save me a lot of clicks by going straight to the set I'm looking for and skipping over a results page.

Gravatar
By in United States,

I noticed that with searching for news articles, nothing pops up... just parts, minifigs, and sets...
You have to go to the search results page...

Edit: Ok, thanks Huw. Great Update.

Gravatar
By in United Kingdom,

That's always been the case, and always will be.

Gravatar
By in United States,

Great feature. Tried Haunted (the full word, not just 'hau') and got exactly what I wanted. The images are a huge benefit in helping to select the right item after searching.

Regarding spaces, I don't think spaces should be ignored, either in set names or in search terms.

Are you just doing raw SQL queries when searching or do you get to control how terms are indexed and how search terms are processed?

I've only really done searching with Apache Lucene and haven't done any sophisticated searching with straight SQL. With Lucene (I'm going off of memory here), you could define how to analyze values for storage in the index. For something like X-Wing, you could store that x-wing and xwing were both valid search terms for the item, if you wished. That was under your control. If you wanted to merge terms separated by a space you could but that wasn't something I ever did (due to the 'sand' problems discussed here) and I'm not aware of that being commonly done.

For the search terms themselves, you could also configure how those were processed. If you wished, you could treat X-Wing the same as xwing if you knew that's how X-Wing was processed when it was indexed. While you could concatenate search terms by space if you wanted, that's not something I encountered, and you'd instead treat space as a search term separator.

A search for 'sand' would thus never match a result that had a word ending in 's' followed by 'and', because when the set Gates and Fences, say, was indexed, the terms wouldn't have been concatenated together.

For something like X-Wing, if you merge strings separated by dash and then lowercase, both when processing search terms from the user and processing entries you find in the DB, that might work. 'x' and 'wing' would still both match it also.

I haven't tried to do this in straight SQL before, and I'm sure there are lots of complexities I'm unaware of here with the Brickset DB. But if there's some way not to concatenate terms to avoid the 'sand' issue, I think that would be really valuable and would make the search more consistent with other search experiences online.

Thanks.

Gravatar
By in United Kingdom,

Spaces are no longer ignored, it caused more problems than it solved!

^ Thanks for your insight. I'll look into it.

Gravatar
By in Australia,

Hi Huw, nice update on the search.
I'm not sure which database platform you are using, but perhaps you could implement SOUNDEX into the search that might help people find items even if their spelling is a bit off. Just a thought...

Gravatar
By in United Kingdom,

Soundex only really works at a word level and often set names have multiple words which complicates things.

Gravatar
By in Italy,

After a bunch of days, some extra comments.

It seems that the improvements are not "free of charge": I noticed a visible slowdown of the website. Nothing tragic, but perceivable.

About search improvements, I was looking for a part in the database, and I found some annoying behaviours:
- Every time I repeat a search, the "type" dropdown menu resets to "All". It would be much more comfortable if it would remember the choice until I close the page.
- The preview include suggestions for the same brick in different colours, so if I want to search, for example, modified 2x1 plate, among suggestions I obtain "2x1 plate (red)", "2x1 plate (blue)", "2x1 plate (green)", ... It would be better to present as a result design id's only (excluding colours, decorations, etc.).
- The main limit to the search in general (not only the preview) is that it tries to match a complete string and not any single words (offering results where the greater numbers of words match or match better). For example if I search for part "4550017: Plate 1X1 W. Holder" using the search string "Plate Holder", the result is a no match, while the expected result is all the plates with an holder.

I hope the examples are clear.

PS: should I post these requests in the form too?

Gravatar
By in Italy,

Another example of mismatch in the search engine.

I bought the 2009 10th anniversary Darth Vader's Tie Fighter Advanced, and I wanted to compare it with all the Tie Fighter Advanced versions released until now.
So I searched for "Tie Fighter Advanced" among sets, but I obtained no results. So I tried with "Tie Fighter" only ("I will select later...". PS: checkboxes for fine selection would help) and I noticed that set 75150 (the last including the tie fighter advanced) was missing. The name of the set is "Vader's TIE Advanced ...", so it is a comprehensible behaviour.
So I thought to access the page of the 8017, almost sure that there I would have found a "tie fighter advanced" tag that I could have used for search. Even bad, neither the "Tie fighter" tag was available.
At the end of all this, I was not able to made a search that ensure me I was looking for ALL the Tie Fighter Advanced released until today.

Not sure how this problem could be solved. Maybe improving tag assignment, using user reports and being very careful when a new set is added to the database.

Return to home page »