Response to survey comments, part 2
Posted by Huw,
In response to our recent user survey, there were a lot of comments about the reviews that we post on the home page. Clearly they are something you feel passionate about but it seems that not everyone is happy with them and it's also apparent from your differing viewpoints that we will never be able to please everyone.
One of the most common complaints was that we don't post enough of them. So, how many do we post? This year we have posted 123. It's day 288 of the year today which means we have posted one every 2.3 days, or an average of 2.9 a week. Surely that's enough? If we posted one a day the home page would be flooded with them potentially alienating those looking for news and not reviews. So, personally, I think we have the number about right and I do not plan to increase our output. Apart from anything else, doing so has the potential to reduce quality given the time it takes to create them.
Some of you suggested that we don't post them quickly enough after a set is released, others say we shouldn't review sets until they are released.
People want us to review older sets, cheaper sets, a wider variety of sets, sets we don't like; some think they are too short, others say they are too long.
Some say they don't have enough pictures, or poor quality pictures, or too many pictures which 'spoils the surprise' when they come to build it. Apparently we should also produce videos for our reviews too.
I came in for of criticism for not applying stickers and also for reviewing sets where I don't know enough about the source material.
Apparently our reviews are too subjective. Surely that's the whole point of a review, to give one's opinion of the set?
Finally, someone even said they don't want us to post any reviews because they like to make their own minds up.
So, as you can see we aren't doing anything right so we might as well give up...
Actually, we will continue much as we are. We will take some of the comments on board, in particular about reviewing a wider variety of sets, those that don't necessarily interest us personally and perhaps to help achieve that we will recruit more reviewers. I will also apply stickers where it's practical to do so and also try and ensure that reviewers of licenced sets are familiar with the source material, or at least do their research beforehand. We will endeavour to post reviews in a timely fashion but often it's simply not possible when dozens of sets are released at once. For instance, we could have divided the recent glut of Star Wars sets among more people, but readers are then likely to find that some reviews are better than others, which would provoke further criticism.
As for the other points raised, we can't really do much about them given your opposing viewpoints!
Get in touch via the contact form if you are interested in becoming a reviewer. Successful applicants will be over 18, able to write eloquently and take high quality images, and will live in the UK so that postage costs are not prohibitive if the need arises to send sets out for review. Let us know which themes you would be interested in reviewing too.
If you don't like our reviews you are not obliged to read them or even see them, if you modify your news preferences.
Tomorrow I will take a look at comments relating to our news coverage.
23 likes
56 comments on this article
Jangbricks on YouTube pretty much has video reviews sewn up so probably not much point repeating here, perhaps a tie in with Jang may satisfy?
^ That's why we haven't -- there's no point competing with the mighty Jang!
What sort of tie-in did you have in mind? As it is he posts links to his reviews on the set details pages so they are viewable here already.
Keep doing what you're doing, I love this page and it's a great source for all Lego fans!
Huw, for sets send out for review, are we able keep it? Or have to return it back once done... too bad I dont live in the UK :(
I would love to write some reviews for brickset. I buy a lot of smaller Lego sets often for stress relief, and while I'm not the best photographer, we have high quality cameras in my house and I can manage photographing a small model easily.
Also, if you're interested in Lego game news, I would be more than happy to provide articles for those. I don't really have time to offer a review, as they take a lot of time to play through and I don't buy every one, but I could do a write up on each game upon its release detailing the critical consensus and how it's differs from other Lego games.
> So, as you can see we aren't doing anything right so we might as well give up...
Well, that's one way to see it ;-) I prefer to think that most of us are happy with the way it is and a few people have complaints. If a reader is happy with the current reviews then that reader probably didn't have any comments to add for that section. Silence isn't necessarily apathy.
I forgot to mention I live in the US. But if be buying my own sets, so I don't imagine that'd be an issue?
As for some people wanting more and others less. If it's that polarized I refer back to what I said on the duplo ducks review:
"Could we put tabs on the homepage to appease those that want more news AND those that want purer news? Something like this: http://s9.postimg.org/4jujtpf8t/Brickset.jpg
The categories I have there are pure, brick built Lego; constraction, Duplo, video games, and non liscened/non official Lego. Those could easily be cut down even to two, by lumping everything except official, brick built Lego into one tab."
Maybe it doesnt seem enough reviews because they sometimes come in feast and famine? Maybe target 1 review every 3 days?
The quality and quantity of your your reviews are just right, thank you. The only logical explanation for the survey comments is that someone is trolling you. Perhaps it was a bitter fan of a poor quality building blok company?
^^Yes, I believe that was the intent. We'll get a stream of reviews for a week or so and then zip squat for two weeks, and that doesn't work that well.
@Gruffy1987 - While I agree in some respects, we are then liable to receive criticism for not posting them quickly enough. If we had posted one review every three days for the recent Star Wars releases there would still be two reviews to come, forty days after their release!
Also, the range of release dates this year has meant that new sets have become available almost every month, so we have to move on quickly from one theme to the next.
I like the idea of getting a free set in the post, maybe I will bush up on my skills and buy a new camera. I think this is one area where you cant please everyone but I loved the recent who review as it was the first time we had seen the real set. Keep up the good work.
No need to worry about everyone's opinion on reviews. Overall the website delivers high quality posts. Furthermore, a lot of reviews are also done on youtube/eurobricks etc plus brickset-users can add their own reviews. So that almost covers in total everything you'd need to know about a set. If you really want it after having read/seen reviews, go buy it and build and experience it yourself, that's probably the best way to go with it...
The most exclusive sets are reviewed here so those are imo the most special ones also...
@Brick Belt, I'll be very happy to accept reviews for the home page. Send me one of a small set that doesn't take too long to do and we'll see how it goes.
Good pictures don't require a fancy camera. They require knowledge of how to capture a good picture. Lighting and framing are more important.
Well, you always get people with different tastes, but you're doing pretty well at the moment. No major changes needed.
Huw, please do not forget that 90%+ of us are super happy about the way the site is managed. Keep up the great work and the quality reviews.
Yes, I'm happy with the site as is. More than happy even! I can't imagine the amount of work you have done and continue to do in only your spare time! It's truly remarkable.
I think the Brickset team provide a brilliant service of all things Lego related, their reviews are great and photographs are exceptional. If people wanted video reviews could they not just watch YouTube? Also, if people do not like a particular review, or the review is of no particular interest to them, then just don't read it. I'm sure they could find the review they wanted elsewhere on the internet or on YouTube. Again for those that think reviews spoil the element of surprise with a Lego set, just don't read it. Brickset provides a service of ALL things Lego, from news, books, magazines, and Ideas, to historical articles on specific themes etc. They do not just provide reviews, therefore explaining why they only publish one every couple of days. Not to mention the work required to write a review. You can't please everyone, and that's going to happen when Lego is so varied and people have different interests, but reviews are only opinions and are meant to be subjective. I for one think the entire Brickset team should be praised for the work they do, not criticised.
Perhaps you could do a review digest like you have started doing with events.
Eg 'Newest reviews added to the database this week' sort of thing. Then people can follow the links if they want, and highlights that section of the site as well (that I rarely delve in to).
I'm glad you keep most of the posts 100% LEGO official! Keep up the good work!
Thanks for doing this work at Brickset. It's an awesome site.
Yeah Brickset homepage reviews are my favorite online: your rather thorough and not to snarky like some other reviewers. While I don't totally understand why you don't apply stickers, I get that it is your personal preference and I support that (you own the Lego set, you decide if it needs stickers or not.)
Huw, your pessimism is surprising. Surely the whole point of writing a survey is to get honest feedback from the community? I agree that you can only please some of the people some of the time, but the tone of the said article had quite a rudeness about it at times.
In the future I would suggest just publishing the results - perhaps with graphics for clarity - and explaining what action will be taken.
Now I can't please everyone with my response to the feedback...
I certainly didn't intend to be rude, sorry if it came across as such. Dismissive, maybe...
Reviews are most useful when it's for something brand new (or upcoming), like the reviews of the new Star Wars movie sets that came out quickly after they were released, and the recent Doctor Who preview. A review for some random City set that came out six months isn't too interesting or relevant since those interested would've already sought out reviews for it by then.
One thing I'd like to see on Brickset are articles highlighting new parts. I know there's the New Elementary site, but they don't really seem all that focused on what they set out to focus on, so I rarely visit. New parts opens up LEGO to new possibilities and as such are important to many, even if Brickset isn't a MOC-focused site.
Honestly, the reviews are fine as they are. While I often use reviews from other sources, I enjoy reading multiple perspectives, and many of the reviews are good. I can't comprehend why people would want less opinion in them. That's why I read/watch reviews in the first place!
I think you're right, no contortions of script and pic could satisfy that list of complaints. I like them, I prefer them to the long you tube rambles - Jangbricks and the guy who does the stop motion merge films being honourable exceptions. I like the idea of older sets being reviewed, but the real value to Brickset reviews are the links - to brick lists, part types, similar sets, updated sets, previous reviews, alternate builds, construction manuals. If you can't make all the people happy all of the time, stick to your strengths which is your huge database.
I like the reviews as they are.
The only comment I would make is that it does help when the reviewer is more familiar with the source material.
I like the comment above about doing a "Recent Reviews" style article that lists recently added user reviews of sets.
I think the site does a good job as far as reviews go. Honestly, I don't usually come to Brickset looking for reviews, but rather news, which makes me curious for what you will post tomorrow...
@Graysmith: I share your same feelings about TNE; I felt like they started out spectacularly, covering a very niche (and underserved) aspect of the hobby, but lately are doing less to distinguish themselves from every other Lego blog out there. I don't know if I need Brickset to pick up the slack, though.
Similar to what Brick Belt suggested, maybe we could have the tabs, but instead of dividing it by theme, maybe divide it by category, like News, Reviews, etc. The posts on the homepage are already categorized so you could leverage that.
I see nothing wrong with the current way new set reviews are conducted and how frequently they're posted. I think the recent push of people wanting to see a lot more new content than the site can reasonably keep up with, is mostly due to being teased with so many new sets announced for the first half of 2016. However, I don't believe posting more about recent set reviews will fix the problem much. Once consumers catch wind of something before it comes out (or even just as it comes out), it's natural to want to know more about it ASAP. But the anticipation may be ruined if too much is posted/reviewed so soon.
I'd say it's okay for people to wonder for a bit longer instead of get the scoop right away, it keeps us excited in anticipation. The quality of the Brickset review articles has been excellent so far, and the proper time needed to continue creating such reviews deserves respect and a little more leeway.
The recurring gripe I have about reviews is that if something is worth talking about then you should have a picture of it. I'll use the 75104 review as an example. There was text about Kylo Ren's lightsaber in narrative about the minifigure itself -- many people were curious about how it would appear in LEGO form. But then there was no picture of the lightsaber! The reviewer fixed it, and my point is that if you take the time to write about it then you probably should show a picture, too.
And another comment I have is about the site in general. Just be yourself and don't worry about coming off as rude. I chalk a lot of that up to differences in how English is spoken/written around the world. The site reflects the personality of you and your staff, and sites that allow that to shine through are more memorable than ones that keep to a bland, news reporting style.
What about reviewing older sets? I would happily contribute to that.
@Huw I don't think anything you said in the article was rude. I think this just confirms that by writing/doing anything, you are bound to offend someone.
First , I don't think this was rude whatsoever. You were direct and I , for one , greatly appreciate directness. As for the site itself , I think the reviews are just perfect as they are , my only annoyance is how innacurate or plain non-existent the database can be for older sets , which in my opinion is the only problem the site has. Of course I won't be upset if there's nothing you can do to fix it , just pointing out almost as many people said the database had problems and so far you've only brought up the review issue (again to me it's not one). As an addendum , if you need someone to help overhauling the database I have several sets that I can provide part inventories for from the official LEGO instructions , and I can be a good researcher for reviews if such is ever needed. Again , it's good with me if you can't or don't want to change the way the database works and the reviews are fine as they are.
Thank you for your work on the site ,
HK-50
^ I discussed the database related comments here. http://brickset.com/article/17110/our-response-to-your-survey-comments.
Ah , thank you. I didn't see that when it was posted , so I'm glad to see it now. Keep up the good work on the site , huw. And please don't remove the database entirely. Anyway I do understand why you can't do it. It would indeed be too much work. And I do plan to help the Action Plan as much as I possibly could.
Once more , thank you for the wonderful site to all the moderators ,
HK-50
@SprinkleOtter - You got it. If people could increase reviews on older sets that would be very helpful to some. My suggestion would be that people could even at the very least just help find external reviews that have been created already and post links to them in the external reviews portion of Brickset and then anyone who wants to can enjoy.
As has been stated also by others, no point in redoing reviews that have already been created. There's even organizing techniques and plenty of MOC's by the likes of Jangbricks, Brickshow, etc. These kinds of reviews are fun. This showcases to the world what our hobby is about and allows us to communicate with each other using whatever medium we prefer.
All one needs to do is search on youtube and you'll be busy watching all kinds of Lego reviews and ideas for days. It's awesome to see the time and dedication these fellow AFOL's provide to our whole community.
Whoever said your reviews are too subjective should have really thought about that before he said that. I mean think about... He was being subjective by saying that your reviews are subjective. No one can review anything without saying what their opinion is. By the way I think your reviews are great :)!
@TitanArch - The text is often written days before or after the photos have been taken, hence there can be some inconsistencies between the images and the text. However, I appreciate the point and will hopefully be able to correct any omissions quickly in future.
I would love to offer up more reviews - the main difficulty being how to post pictures, which I haven't mastered yet.
I don't post very often but use your database and look at your site daily. I'm sure you don't need me to tell you (Huw et al) that often respondents to surveys will be negative as they have a greater motivation to air their views. I just wanted to say that I didn't respond to the survey but think you guys do a great job and you are my definitive resource for Lego news...please keep up the good work!
I really like the reviews posted here at Brickset. It is the principle reason I visit this site. There are times when the reviews posted are biased towards one theme but that is most likely due to the set release schedule that LEGO has in place.
The varied interests of the principle reviewers is also great which means some themes have specialist review content.
I like writing reviews of my sets and have been working to emulate the writing quality and have clear images similar to those posted here.
I would like to have some of my reviews posted on the Home page. My principle theme is Technics but lately I have been working with Mindstorms.
I would also like to add my voice to the people who've said that the frequency and quality of the reviews are great as they are.
As an adult, I'm smart enough to not be bothered by opinions and tastes different from my own. The photos added to the reviews serve their purpose very well and I look forward to every review as I usually take a quick peek on the website every day when I'm at my work. This is the first website I go to for any news on LEGO and if I want to know more, it is not hard to check Google or YouTube.
Just keep doing a great job!
I think the reviews were fine as but that every once in a while, like once a month they should do an old lego set.
I like the reviews as they are. There are plenty of them, and I think they are current enough. Look at Doctor Who the other day; the review was up while everyone else were still raving about its mere excistence.
Older sets have been covered through retrospectives many times. Actually, retrospectives by guest reviewers was a very good idea, maybe more of those could be a good way of appeasing some people? I'd love to see more myself.
I agree J0rgan, there is nothing wrong with the reviews. I'm actually surprised people posted complains in the first place!
Love the sight - you're doing a great job. Keep it up!
I don't recall exactly but I thought the survey was asking for things that could be improved. I wouldn't necessarily call answers to that question "complaints". I love Brickset but it is still not quite the perfect site for me personally. Of course people are going to want different things and so many suggestions are not going to be actionable. A consistent theme across responses is obviously more interesting.
Regarding "If we posted one a day the home page would be flooded with them potentially alienating those looking for news and not reviews": I am sure I am not alone in feeling that threshold has not been reached yet. In any case, I think it is based on a faulty premise that the structure of the site needs to be a one-size-fits-all single home page stream of all content. I believe that there are some customization options already available, and more of that (or even a more radical UX) might allow for increased content across many topics without alienating anyone.
Is there any way in which a new "Save as Draft" button could be added at the bottom of the current Review Pane.
The idea is to have some way of parking a review that it not yet complete and therefore not available for general release.
You sound a bit deflated @Huw about the site. I love Brickset and of course you can't appeal to every individual and there’s always going to be criticism (hopefully constructive). I think just take them on the chin and implement the suggestions that YOU feel would benefit YOUR site. Some of the comments sound petty and personal, which I think is a downside of the ‘feedback’ survey. I always want more of everything because I LOVE LEGO! I’m trying to think how I can help with reviews – I love the Ninjago theme which I feel could add to your site, although I don’t know whether I have a good-enough camera set-up to satisfy the need for high-quality photos.
Honestly the set reviews are the best I've found on the internet. The format is good, the photography is top notch and the tone is pleasant, unlike a certain site Star Wars Lego site I used to frequent. Don't fix what isn't broken!
If I had one criticism, it would be to PLEASE stop posting The Force Awakens rumours in the reviews. I've had to stop reading them. It's one thing to see the Lego sets, that doesn't mean I want plot spoilers.
@JonMarten - To be fair, we did post an article warning of potential spoilers in advance. I think they are an inevitability; a key aspect of the success of a licensed set is its faithfulness to the source material, so we have to acknowledge it.
I enjoy your sense of humour :) You cannot, and never will, please everyone. So while this site remains, in the main, a labour of love, improving it is a great thing, but never to the point you no longer enjoying hosting.
Personally, I think you do a fantastic job. I enjoy the reviews and the entire site in general. Basically, I think that both comments and stats indicate some polishing is required, at your discretion and in consideration of feedback, however, to the vast majority, you and your cohorts have developed an excellent site, as evidenced by those who are repeat visitors, including myself.
So, in light of the above, I just wanted to say, Thank you.
Just gotta say thanks so much for taking all the feedback on board and adding all those images! It sounds like the sort of time-consuming work many others would just make an excuse about and put off but you evidently care enough about the userbase to get on this quickly!