2016 user survey results
Posted by Huw,
The survey has ended and the figures have been collated. Thank you to all 2378 of you that completed it, particularly the 695 who took the time to provide comments and suggestions.
You can view the satisfaction scores for each question after the break, and the change compared to last year's.
The net promoter score, your response to the question 'How likely are you to recommend Brickset to other LEGO fans?' is a healthy 84.2%, up 10% on last year's score, which suggests that overall we are improving.
Later in the week I will address the prominent themes emerging from your comments.
The figure is calculated by subtracting the percentage of scores 1-6 from the percentage of those 7-10.
| Question | 2014 | 2015 | 2016 | Change |
| Quality, relevance and timeliness of the home page news | 91.8% | 89.7% | 86.1% | -3.6% |
| Accuracy of the information in the database | 95.1% | 93.0% | 93.9% | 0.9% |
| Quality of set images in the database | 93.8% | 92.4% | 93.0% | 0.6% |
| Ease of use | 90.7% | 91.6% | 89.8% | -1.8% |
| The set reviews on the home page | 92.0% | 89.9% | 86.6% | -3.3% |
| The design of the site | 92.6% | 92.7% | 91.3% | -1.4% |
| Speed of response | 90.2% | 89.2% | 84.0% | -5.2% |
| The forum | 49.1% | 58.0% | 58.0% | 0% |
I will comment more fully on why I think the home page news/reviews scores have dropped later in the week.
I am not going to worry about the 'speed of response' question as it became apparent that many did not understand it or interpreted it as the speed at which we respond to the contact form and not the speed of the website. I will reword it next year.
The forum still has a low score but despite this I have no intention of changing it. I know I will get shot down for suggesting this, but if you do not like it there is probably a LEGO forum, one that uses more traditional forum software, out there that you will: Eurobricks, for example.
I do not feel that we have to have a forum that suits everyone, it is simply not possible to please you all as I am sure you appreciate. Those that have taken the time to use and learn it love it, and those that have not, don't...
39 likes
29 comments on this article
Is the drop in home page/news reviews greater than the margin of error? The drops are fairly minor relative to the very consistently high averages.
Quality and timeliness continued to drop. Interesting.
I did like the recent article about upcoming retirements, that is useful.
I can't access the survey responses link for some reason. It says, "The form "The 2016 Brickset user survey" is no longer accepting responses.
Try contacting the owner of the form if you think that this is a mistake."
I had the same issue
If nothing else, those database related scores are pretty sky high and so they should be! I personally think the homepage news deserves even more, especially with the topical articles and opinion pieces.
OK two things are bothering me:
Firstly I have taken the time to use and learn the forum and I don't love it. Somethings are good, some are not so good but overall it doesn't get a 8-10 for me. But I didn't give it a 8-10 last year either so I am really surprised by the large drop. What has changed over the last year because its not the forum?
Secondly why is the score for the forum expressed as 58% and not 58.0% when other scores are written as 93.0% and 84.0%
The second point is bothering me a lot more than the first, this might say more about me than anything else.
Same "survey is no longer accepting responses" issue over here...
I don't suppose, for the forum thing, that the software calculated "I don't use the forum" as a 0 score? Just a wild guess, but if there are more people not using it, that might be a factor...
That looks like pretty interesting data, but I don't actually know what a bit of it means... I will wait and see for the analysis.
I'm curious as to why there's a change (negatively) from 2014 and this year. I'm looking forward to a more in-depth analysis.
Brickset is incredibly informative and useful. I, for one, appreciate Brickset. Any survey has people who do not give the questions thought and/or are just plain destructive instead of constructive. Do I love everything about Brickset? No. But I like a lot about it.
That Brickset recommendation number should be higher, in my opinion. I've been using the site for years, and have found it to be fast, reliable, and informative.
I would suggest perhaps more set reviews, but other than that, the format and information on the site + the articles are very accessible and a great Lego fan resource.
For my part: I really love Brickset, but other sites are stepping up too and I have lots of information on theses sites before they appear on here. (Promobricks for example, a German page.) They seem to be more active on Facebook/Twitter as well. Not just linking new articles, but also adding content exclusive to Twitter or Facebook.
Your reviews/database are world class though.
Sorry about that -- seems you can't share responses with the new version of Google Forms without making everyone a collaborator.
Here are screenshots of the numerical questions:
http://images.brickset.com/news/survey-top.jpg
http://images.brickset.com/news/survey-bottom.jpg
I'll publish the comments as a PDF is due course, once I've finished commenting on a few of them.
I would agree that, except for the forum question (and maybe the 'speed of response' one), these numbers to me seem to indicate that everything is as it was last year. No improvements, but also no drops in popularity. Any differences to last year's scores are likely just random (or in the margin of error)
@Huw I'm not sure this comment about the forum is true at all in many cases.
Those that have taken the time to use and learn it love it, and those that haven't, don't...
The forum isn't difficult to use it at all, I think Eurobricks is more complex to navigate around. There isn't much activity in more than 5-10 topics at any one time on the brickset forum and it's pretty clear what is either a new topic or has new content. I think it might be more a content issue. It seems to be the antithesis of Eurobricks, so as AFOLs we have two extreme choices for a type of international forum.
CCC, sorry, my mistake. The calculation I've used is subtract the 1-6s from the 7-10s. not 1-7 and 8-10 as stated.
695 responses:
1-6 -- 146 -- 21.0%
7-10 -- 549 -- 79.0%
58.0%.
I've edited the article.
Huw is there a prize for working out why the forum has dropped over twenty percent?
If you look at last years numbers 78.1% gave the forum 7-10 before you take anything away, this year it was 79.0% the numbers have gone up.
The forum hasn't dropped 20% its a error in the calculation of previous years?
You are right! I did not subtract the 1-6 percentage the last two years. I make it
2015: 1078 responses, [1-6: 236 = 21.0%], [7-10: 842 = 79.0%] = 58.0%
2014: 1748 responses, [1-6: 445 = 25.5%], [7-10: 1303 = 74.5%] = 49.1%
Now why didn't you notice that last year :)
Article corrected. Thank you.
Tend to agree about the forum being a bit of a Marmite (i.e. love-it-or-hate-it) thing. People who want a massive amount of activity every day, or threads to stay on topic, or other kinds of content that we don't really have here (e.g. Eurobricks' extensive RPG threads) will probably drift off. I think it's nice that there's such different kinds of forum for Lego fans.
I also get the impression (from more general chatter over time) that a lot of people are confused by the fact you have to register separately for it than for commenting on the main site, but that never seemed illogical to me because I've been on so many forums in the past. Once you get past that it's no worse than any other forum I've come across in terms of ease of use; I'd guess that a lot of people who are alienated by the interface might just not be used to forums in general, and would be equally put off by other Lego boards.
About that prize :)
that's pretty funny to run a survey and then say you don't plan to improve on the only area getting a mixed answer :/
I think one big improvement to the forum could be to reduce the number of posts by page. some members tend to post lots of pictures, and 50 posts with pictures take ages to load. maybe that's actually the reason of the low activity on the photography thread.
^ That's because it's impossible to improve it.
Leave it as is: those that are happy with it remain happy.
Change it to something else: alienate them with no guarantee of making it any more attractive to others.
Interesting, will wait for the analysis to see why.
I "discovered" Brickset 3 years ago, and since then I´ve checked the site a few times every month. I think the news is better, that there are more reviews with good pics of the sets and much easier to use the site, than it was back in 2013-14. Brickset is pretty much the best site I know - even though you´re not logged in, there isn´t much ads, and it never lags or suffer by any technological problems. I wonder why most of the results on the survey has dropped down a bit.
However the new improvements of the site was made during the survey, so using the site now is a little easier than it was a month ago.
Thanks for making a survey to us users, Huw. (:
Brickset and Jangbricks youtube channel are the only Lego sites i use almost everyday. Those two together, I can get all the lego set info i could ever want!
I dont know what i would do if i didn't have Brickset to come to. My collection would be a lot smaller if it wasnt for many of the features here. Half the sets i get are from seeing the Amazon discounts front and center. Plus, that helps the site too! Glad to do my small part.
I'm not a fan of the forum, but, I agree that it shouldn't be changed.
I'm used to forums built on the likes of v-bulletin, so Bricksets forum is a bit alien to me so I don't use it very often.
Plenty of folk do use it actively, so I think it's right to leave it alone.
Well, Huw, the results look satisfactory to me. I am glad you are rewording the question about speed of response next year, as I had trouble answering that one. The only reason why I didn't answer the forum question is because I'm not old enough to use it yet.
"Speed and responsiveness of the website"
People keep talking about the forum, but the results went up from 2014, so it is likely a step in the right direction. It is still lower than everywhere else, but maybe that isn't the primary purpose of the website? Either way, I like the forum well enough, but I agree about the pictures taking forever to load and ruins most pages for me (so I just quit looking) when there are too many on one page.