User survey results
Posted by Huw,
Thank you all 3676 of you that participated in our user survey: that's nearly twice as many as did so last year. You can now view a summary of responses and also download a PDF of the comments.
To get a measure of customer satisfaction and to compare this year's results with last year's, I have used the same calculation method as before, which is to determine the percentage of 'satisfied' scores, 7-10, for each question.
You will find the table, this year's net promoter score, and the name of the winner of the Birds set below.
| Question | 2014 | 2015 | Difference |
| Quality, relevance and timeliness of the home page news | 91.8% | 89.7% | -2.1% |
| Accuracy of the information in the database | 95.1% | 93.0% | -2.1% |
| Quality of set images in the database | 93.8% | 92.4% | -1.4% |
| Ease of use | 90.7% | 91.6% | 0.9% |
| The set reviews on the home page | 92.0% | 89.9% | -2.1% |
| The design of the site | 92.6% | 92.7% | 0.1% |
| Speed of response | 90.2% | 89.2% | -1% |
| The forum | 74.5% | 78.1% | 3.6% |
Most scores have gone in the wrong direction, although not dramatically and none are a major cause for concern.
Out net promoter score this year, calculated from the last question on the survey using the method outlined here is 74.4%, 2.4% lower than last year's score of 76.8%, but still an impressive number.
The lucky winner of the Ideas Birds set is chriswhyte101. Congratulations Chris, I will be in touch to get your snail-mail address.
I will go through the comments and post a summary of the recurring themes and an action plan to address them next week.
15 likes
18 comments on this article
From skimming through the comments, it looks like a lot of people want more reviews, from more themes, on all sizes of sets, and also more general articles and articles on older sets.
I know I haven't done much for Brickset recently, so I'll have a think and then ask the boss aka Huw what he thinks of my review/article ideas :-)
Good to see that the majority still think Brickset is brill, a sentiment with which I wholeheartedly concur!
I use Brickset mainly as a database of my collection and sets I have on my wanted list. -And a source for news, sometimes, quite often frankly, I get the news first from other sites, but I`m happy with BS nevertheless.. Reviews do I read on Eurobricks. Keep up the work Team BrickSet!
Some of the comments do make for interesting reading...a bit of a skim through does seem to suggest the majority are happy with what's here but just want more of the same.
I find it funny that there are quite a few that seem to ask for functions that the site already has...maybe a detailed how to use Brickiest article is in order ;-)
A lot of comments ask for ability to know what pieces each user has based on what sets they have. I would direct them to Rebrickable.com which is a linked site on the set pages. You can easily make a list of your opened sets and then see what parts you have and even add more loose parts if you wish.
I love this site,but your reviews are sooooooooo short.
From the comments on the original thread, I know quite a few responders only scored from 1 - 5, because 6 -10 didn't show on their phone screens. That could have skewed the results a bit.
Huw, the best thing is that you obtain almost double participants this year!!!!
I'll discuss these emerging themes more in my followup post but, briefly, about reviews:
People want more, ^^ thinks they are too short, some say we only review the same types of sets (e.g. exclusives and expensive ones).
It's obviously hard to please everyone but say we were to publish more. When would there be too many? One a day? one every couple of days? As it is we try and limit ourselves to 2 or 3 a week otherwise the 'news' would be flooded with them and they'd disappear off the home page too quickly.
Even if we posted one a day that would only be 365 reviews a year which, this year, would cover just half of the products LEGO has launched.
Writing reviews is time consuming and I actually find it quite difficult to do so, even for sets I am interested in. I would find it very hard and tedious reviewing those I'm not. Obviously we can (and do) have other reviewers but it can be hard to maintain the quality of narrative and photography that readers expect when we do so.
^^ I'd be interested to know which ones you think are too short and in what way they should be lengthened.
Finally, don't forget that anybody can write a review on any set they like which is then visible on the set details pages. Maybe a way to go some way to increase the number and diversity of reviews visible on the home page would be to publish a daily summary of new ones in the news feed.
^That's not a bad idea, Huw, to list a summary of the newest reviews available on the site. Most sets in the database have at least one user-provided review that should at least get people started in the right direction if they're looking for information or want some help making a purchase decision, etc.
Knowing that their review might get mentioned on the front page could encourage us on the sidelines to start adding more reviews, too. This is a double-edged sword, though: you might get a deluge of sophmoric contributions rather than a quantity of useful reviews. But it might be worth the risk if the major complaint about the site is the lack of total set reviews...
Consider it implemented... I'll restrict it to showing reviews from gold/silver/bronze-rated and trusted reviewers for now.
And here I am thinking your reviews are impressively informative - not too long, and definitely not too short.
Brickset Is Awesome - Thanks For Your Fantastic Efforts.
I found the reviews quite long and rarely have time to read them through. I'd like to hear more personal opinions. You could share a link to the database - then there wouldn't be needed to tell everything about price and piece count. I don't find it helpful if you write like five sentences about a standard battle droid.
Good work anyways :) this was just my view
English isn't my home language and that's why I don't always have enough patience to read long stories, haha.
I'm one of the responders who wanted more reviews and long-form articles (and due to a technical glitch I may have accidentally asked for it twice!).
I think you could maybe even hand-pick well written user reviews of recent sets (less than three months old) to be formatted and placed in the news feed alongside regular Brickset goings-on.
Thinking back on my response to the survey, maybe one of the reasons that compelled me to ask for more content was those times where the Brickset news feed would have nothing new to read for a few days, which really annoyed me. Say you sit down to breakfast and you want to check out what's new in Lego land while you mung on a muffin, but the top of the feed still has a story about leaked set images from three days ago, and the images were removed at the request of The Lego Group.
Yet we come to this week, and I count nearly 10 new posts in the last day-and-a-bit and I haven't had a chance to read them all yet; breakfast for today will be great! Pumping out the timely stuff like the Doctor Who review and fresh news is awesome. But sometimes you need filler for the slow news days to appease those of us who might be addicted and habitual Brickset refresh button hitters :)
Getting the balance right between too much news and not enough is tricky.
Some sites fill the slow days with articles of little interest to most. Personally I think it better to post nothing rather than a load of drivel.
Maybe introduce a reviewer system where people who want to be front-page reviewers get in touch, then write a certain amount of reviews judged by the moderators and Huw, and then go ahead and write reviews for Brickset if quality and content is good. Kind of like applying for a reviewer's job, or like the Reviewer's Academy on Eurobricks.
@ Huw - You hit the nail on the head. I think it would be better for you to post nothing than drivel. On Sundays, I am desperate for news, but it appears you all take at least one day a week off! I would rather be looking forward to news, than disappointed in it's quality.
Agreed. Better to post no news than to post junk news.
Here's an idea - as you are implementing this crowd-sourced user reviews on the home page feature, it would be great to be able to identify which Themes are of interest. This could be a setting in the user profile. Say, for example, I'm only interested in reading reviews on Ideas, Star Wars, and Batman sets - I could choose only those and only those reviews would appear in my newsfeed. It's almost like taking the Flipboard model of custom newsfeeds and applying it to LEGO reviews.
I also agree that only reviews for sets that are currently available and have been published in the last X months is a good way to prevent old reviews from cluttering the home page.