Brickset review quality

 

  • Brickset review quality

    Posted by Huw, 26 Oct 2010 15:22. Filed under Brickset.

    The user survey is still running: it's been a great success, with around 750 responses so far. I've been reading the comments and there are some good ideas and suggestions being made. One common complaint, which if I'm honest comes as no surprise as it was also complained about last year, is the quality of the reviews.

    It's a difficult problem to solve given that there are over 26,000 reviews in the database which are being added to at a rate of 10-20 a day. Clearly, weeding out the 'unhelpful' ones is not something I or another individual can do easily: besides, what qualifies me to decide if a review is 'unhelpful' or not? Different people have different opinions.

    So, I've implemented what I hope is going to be a quick-fix which I hope will help weed out the poor ones: I've added an algorithm to the reviews database which will effectively delete them when they have been marked as unhelpful by enough people. If a review has over 5 ratings and more than half of them are 'unhelpful' then the review won't be displayed any more.

    For this to work it needs you help: Please, if you read a review and you think it's not worth the paper it's written on, so to speak, then mark it as unhelpful. If enough people do this, the poor ones will be weeded out.

    I'm not 100% convinced by algorithm is perfect, and I'm sure over time it will need to be tweaked, but that's easily done once enough ratings have been made. So, play your part: rate every review you read!

    Update: I've made a couple of tweaks following your requests: the rating area is now not so prominent and the alert box no longer appears, just a discreet message under the radio buttons.


    54 comments, latest one posted 31 months ago. (Commenting has ended on this article)
    492 views


 
 

Comments

Posted by Alemas in Germany, 26 Oct 2010 15:52

Roger that, I'm on it! *Rushes out*

EDIT: Woah, the place where you rate reviews changed right before me! Cool...
EDIT #2: Just a suggestion: The "rating boxes" detract WAY too much from the review! They're an eyesore...
EDIT #3: If a review is "too unhelpful", will it be deleted forever, or just hidden? And will it affect the overall rating of the set?
EDIT #4: Done Page 1 of the reviews. Now... just 878 more to go. *Faints* :P

Posted by Doctor Leg O'Brick in United Kingdom, 26 Oct 2010 16:11

I just saw this. There go 4 hours of one of my reviews. But i don't think this a bad, wish i had come up with the idea myself................Yep, its gone...........i think the bricklists need some work, like leaked info on new sets, even profiles have data thats band here. Is their going to be a data base wipe like you did years ago?

Posted by Oboe in United States, 26 Oct 2010 16:12

How about a way to sort the comments by how helpful they are to others. I believe that is how amazon sorts the comments there unless you opt to change it.

Posted by natro220 in United States, 26 Oct 2010 16:13

Thank you Huw!!! Awesome idea, one of my biggest complaints. Your site just gets better and better! I never hestitate to mark a review as unhelpful if I think so...now I'll have even more motivation to do so! : )

Posted by Alemas in Germany, 26 Oct 2010 17:19

Huw, those rating boxes detract too much from the actual review! They are too highlightet!

Posted by stu241273 in United Kingdom, 26 Oct 2010 17:34

Huw, the little popup box that says "thank you, your rating has been recorded" drives me crazy. It would be much better if it was less intrusive, I might be inclined to rate more reviews ;)

Posted by Huw in United Kingdom, 26 Oct 2010 17:53

Good point Stu: I've made some changes on my development machine which I'm just testing to improve this. I'll upload tomorrow.

Posted by Huffy1968 in United Kingdom, 26 Oct 2010 18:01

I think this is an excellent idea, simply because it will help to remove all those old, short and badly written reviews from members who clearly did not own the sets but wanted to be top of the (now thankfully defunct) reviewer league table. They know who they are...

Posted by Town in United States, 26 Oct 2010 18:08

I think that 3 out of 5 people saying a review isn't helpful is too harsh. I think the rule should be enforced when 10 or 8 people have "rated" the review.

Couldn't the least helpful reviews just be sorted to be listed at the end of all of the reviews?

Alemas; I am wondering that myself, will a deleted or hidden review affect the overall rating of the set?

I haven't reviewed anything in a while, but is it still not possible to edit a review once you've submitted it?
Also, is it still not possible to add ratings without writing a review?

Maybe I should do the user survey...

Posted by Huw in United Kingdom, 26 Oct 2010 18:14

> I think that 3 out of 5 people saying a review isn't helpful is too harsh

It's actually 3 out of 6. At the moment number of ratings are low: I think the highest is 33 and the average is something less than 10 so maybe when these increase the formula can be tweaked.

> I am wondering that myself, will a deleted or hidden review affect the overall rating of the set?

No, they will be hidden from that calculation.

If this all works I will reinstate the top reviewers system, which will reward people who write quality reviews, not quantity of reviews.

Posted by benjadude11 in United States, 26 Oct 2010 18:15

what if a review is just provocative, not unhelpful? Cough cough, three good reasons, at this link- http://www.brickset.com/reviews/?set=7136-1

Posted by Ypres in Canada, 26 Oct 2010 18:54

I hate how bias some reviews are... so many people give 5/5 to a set without understanding what a perfect score is. Things like that are why I stopped taking Gamespot reviews seriously.

I hate how some people also just decide to randomly mark reviews as unhelpful... and as example I thought my review of Cad Bane's Speeder was more than fair, although clearly someone thought otherwise. If only there were some way, rather than mark as unuseful, to submit complaints (giving a legit reason) about reviews that are not of brickset quality so they can be respectfully removed. I know that is probably impossible, but sometimes I like to know why one person reading my review didn't like it.

Posted by jake9 in United States, 26 Oct 2010 21:10

Honestly, it doesn't sound stringent enough. If there are really thousands of reviews, at this rate, it'll take a long time for the bad reviews to get weeded out. I think 2 out of 3 would get results faster.

Posted by sudorum in United States, 26 Oct 2010 22:08

I have always been slightly annoyed by the reviewing system. Highly detailed and perfectly relevant reviews have been marked as unhelpful for no apparent reason. This needs to be fixed.

Posted by Orthobotrex in Philippines, 27 Oct 2010 01:00

Good day!
I really don't know if all the reviews here are being done by AFOLs. If not, then you might have to add a review by kids, though I forgot already if being an adult is a prerequisite for joining this group. You might have a parent browsing for stuff to give his kid, and an otherwise strictly technical review might wipe out the merits of a particular set. An adult might expect something more challenging for a build, while a kid might simply want to have fun playing with his or her stuff. In this regard, maybe there should be a classification of the reviewer and why he is doing the review.
Otherwise, there might be a language barrier involved, and the reviewer might still be trying his or her best to write something of worth but it comes out as unacceptable. Also, maybe you guys could decide on an expanded criteria as to how we should rate the sets. Just don't include value for money, as LEGO sets are priced differently depending on where you come from.

Posted by CharlieTheFunEngine in Australia, 27 Oct 2010 02:41

Not happy. Must say that this is one change that I strongly disagree with. Now my Fantasy Era review is gone and two of my City reviews are on the verge of disappearing. Sigh...

Posted by Doctor Leg O'Brick in United Kingdom, 27 Oct 2010 03:25

@TinyTurbosCityAndToyStory, im not happy too. My funny review about a XL gray baseplate was REMOVED because 8 out of 3 people think its rubbish. I tell you, what are you going to say about a baseplate?? "It has no bricks and no minifigures and no slopes and no flag poles and no swords and no guns and no...." This is what i mean, what do people WANT from me and this baseplate???? And i agree with all who have commented, how do we know if someone has just randomly press no, or people rating it a 5 bricks all the time when it should be 4, 3, 2, 1??? Yes, all my reviews but a city one are rated a five because i do mean it, i don't think i have brought a set in three years that has not made me want to destroy it or feed it to my cats. And you know why??? This place. Brickset.

Oh, TinyTurbosCityAndToyStory, that castle review was good.

Posted by Ulli in Germany, 27 Oct 2010 04:28

I think that no review should be deleted by an algorithm!!
They should better be sorted by the ratings. Furthermore there should be a short (60-120 signs) comment field, where a user should add in a short comment if he think that a rating is unuseful! That will help the author of the rating to improve! And this will help to get an overall improvement of the quality!
Think about that:
You take the time to write a really good review of a set.
Then, for what reason ever (they always hit "No", they don´t like the set, they don´t like you, they don´t like ratings written by an specific nationality...), the 4 first to read the review are rating it with Unuseful. With the next two ratings as "Useful" it will be deleted (4 out of 6)! Are you ever again spend your time by writing a review?

If you really want to delete certain reviews, this should be done by an admin, who can also consider the comments given with the "Unuseful-Rating"!

Posted by Matthew in United Kingdom, 27 Oct 2010 05:02

ARGHHHHH My review that I spent ages on yesterday has gone :-( What was I supposed to say about a manta warrior keyring that I was given for free by Lego Customer service?
Do they get deleted or just hidden?

Posted by Huw in United Kingdom, 27 Oct 2010 06:18

Just to be clear, nothing is deleted, it's just not displayed any more, so if the need arises to tweak the calculation they will appear again if they meet new criteria.

One thing I can easily do is get the system to show them when viewing 'all reviews by [username]' but not when viewing the reviews for specific sets. That way if there's a need to appeal against one being hidden it can still be read and the review ID determined for me or another moderator to reinstate it. I'll make that change later.

@Ulli: "If you really want to delete certain reviews, this should be done by an admin, who can also consider the comments given with the "Unuseful-Rating"!"

In an ideal world I would agree but there's just too many of them and only so many hours in the day that I can spend doing it. My time is probably better spent enhancing the site or maintaining the database. When I read the new reviews as they come in, some stand out as rubbish and some I can't decide if they are helpful or not. But someone has taken the time to write them and someone else may find it helpful so it's very difficult to delete them at that point. I therefore let them through and then you, the community, can decide if it's helpful.

@sudorum: "I have always been slightly annoyed by the reviewing system. Highly detailed and perfectly relevant reviews have been marked as unhelpful for no apparent reason. This needs to be fixed. ".

Do you have any suggestions as how to do this? I am hopeful that if enough people rate reviews, the majority view will prevail and the odd rogue unhelpful rating will not have an impact. If you have specific examples I can check in the database and find out if there are individuals who are persistently marking them as unhelpful, remove their ratings and disable their account.

As I said, this method probably isn't perfect and I'm open to ideas as to how the dross can be removed. Review quality was mentioned probably more than any other thing in the survey, so it is something that needs attending to.

Posted by atkinsar in United Kingdom, 27 Oct 2010 06:26

I can see why some people may be upset if they've spent a long time writing a review, only to have it deleted (or at least not shown) because others vote it 'not useful'. On the other hand, I do think the quality of the reviews in general could be better.

I don't think Huw and the other site contributors have got time to moderate each and every review, they do enough work on Brickset as it is. An alternative solution, which I think a few people have already hinted at, would be to push the most helpful reviews to the top of the reviews on each set details page. You could use an algorithm to determine this, so once a review has reached a minimum of 5 ratings, you could just score it on a percentage of 'useful' ratings, so 5 out of 5 being 100% and 2 out of 5 being 40%. Then simply order the reviews by this percentage so that the highest are at the top. All reviews would still display ALL reviews, regardless of percentage score, but the lowest scoring would all be at the bottom.

If a review is really bad, then perhaps there could be a 'report this review' link, that allows you to state your reasons. Admins could then review the reported reviews and make a final decision as to whether the review should be deleted entirely (perhaps first giving the reviewer a chance to edit thier review to improve it?).

A few people have mentioned about specific reviews being deleted whilst making a comment along the lines of "it's a small set, what else can I say about it?", to which my response would be, if you haven't got much to say about a set, why review it?

Posted by Alemas in Germany, 27 Oct 2010 06:39

Ha, there's a review on the verge of "deletion": http://www.brickset.com/reviews/?ID=36663 First review of 7op89(my friend at LEGO.com), and it's already close to "deletion". This could make him think that his reviews are terrible right from the outset!

Posted by DrathMaul1997 in United Kingdom, 27 Oct 2010 07:36

While we're on the subject of reviews, I wrote a review a while ago and it wasn't very good. In fact, it was awful and it got 7 'not helpfuls'. So when the the option to edit your reviews come out I edited it. It is now (hopefully) quite helpful, however the 'not helpfuls' remained...

Could we thus have all the not helpfuls/helpfuls removed if you edit? However this WOULD collide with the delete reviews thing, people could edit their reviews and get rid of he 'not helpfuls'...

Posted by Huw in United Kingdom, 27 Oct 2010 08:13

@atkinsar, yes I can sort them with the most helpful at the top: that calculation is already done. On pages such as this, http://www.brickset.com/reviews/?set=4736-1, it may be the right thing to do. However it would mean that new reviews, which have't yet been rated, will be at the bottom and therefore may never get read and rated.

This page, http://www.brickset.com/detail/?Set=4736-1, the review tab, shows the 5 latest. I haven't only shown helpful ones there, again, for the reason that new ones won't get read and rated and therefore will never reach helpful status.

"Report this review" is a possibility but I don't want to increase our workload unduly.

It's a difficult thing to get just right, but do keep the suggestions coming!

@DrathMaul1997, I've been wondering whether to wipe ratings when a review is edited, and if I change the 'reviews by [username]' to indicate when they have been rated as unhelpful it will be easy for authors to identify them. However, the reasons why might not be obvious, and clearly if they thought it was OK when they submitted it initially they are unlikely to be able to assess why themselves, so it might not really solve the problem.

What do others think?

Posted by atkinsar in United Kingdom, 27 Oct 2010 08:24

Huw, the Latest Reviews will still show new ones across many sets, so people can rate them there. To combat the problem of not seeing them on the set dteails page, you could add a new link at the top of the Reviews tab, near 'Read All Reviews', that displays new reviews, or perhaps even better reviews not yet rated. In my opinion, it's more important to show the top 5 useful reviews on the set details page than it is to show the 5 latest (which may or may not be useful).

Posted by Doctor Leg O'Brick in United Kingdom, 27 Oct 2010 08:49

Well, i agree with both DrathMaul1997 and atkinsar, very good ideas. As other people are saying, how do we know if someone just randomly gos round finding a member whos spelling might not be that great at spelling, might not say much of the set itself and other things. Its good that you make sure no ones been targeted by other people, thats what made me join. Nice improvements but i think the bricklists need to be sorted out.

Posted by Alemas in Germany, 27 Oct 2010 09:36

Woah, review ratings and reviewing reviews are getting ever more complicated just to improve review quality. :P What I want is SIMPLICITY! All this talk is driving me crazy!

Posted by Alemas in Germany, 27 Oct 2010 09:40

I'm going to review reviews (:P) as they come and when I stumble across any! Reviewing 878 pages of reviews is WAY too tiring! We would need Superman for this...

EDIT: Here are MY reviews: http://www.brickset.com/reviews/PostedBy/?User=Alemas If anyone could review them, please? :) BE WARNED: Some of my older reviews could be REALLY cheesy. :P

Posted by itsaturkey in United States, 27 Oct 2010 09:41

This system is, well, kinda annoying. It's just that there's too many problems with it. First, like Orthobotrex said, kids and AFOLs like sets for different reasons, and also kids might not use capitols, spelling, punctiation (see, i can't even spell that) and that might make an adult (or anyone else) think that it's 'poorly written'. for example, i made a review of an old adventer set (you can read it on my review list) i think it's written well, but theres already two people who say they don't like it. And thats the other thing, (like Docter Leg O'brick said) someone could just make six fake profiles and go around deleating reviews (okay, thats probably not going to happen)
I do think it's a bit unfair for kid's, expecialy smaller kids who havn't done a whole lot of writting yet.
atkinsar- Great idea! although what if someone wanted to be on the top five list so his friends just voted for him?

Posted by jake9 in United States, 27 Oct 2010 11:20

It would be cool if, instead of deleting the review, the systems somehow sends it back to the author for an opportunity to edit the review and re-publish. That way the author has a recourse, and it will encourage reviews are looked at twice by the author.

Posted by Doctor Leg O'Brick in United Kingdom, 27 Oct 2010 12:04

Now you can not rate a review before you log in.

Posted by Huw in United Kingdom, 27 Oct 2010 12:15

^ That's always been the case, I've just hidden the controls now.

You have to be logged in so the system can record your 'vote' against your name, ensuring only one vote per person.

Posted by Krogh in Denmark, 27 Oct 2010 14:31

I think this is a great idea, if the community can be honest when rating reviews, and not just rate them unhelpfull, when the reviewers oppinion differes from ones own. :)

I find myself checking the "unhelpfull" boxes when i read a review that lacks detail or relevent information, i try not to judge spelling errors or other mishaps, since english is not my first language, i am also prone to these things when i review a set. A review like "i bought this set and it is good", simply is not enough, and there are waaaay to many of these reviews, hehe.

I am not good at checking the "helpfull" box, even when reading a quality review, but now that this actually matters, i will try to rate every review i read, bad or good. :)

I really like the idea of a "kids" section in the reviews (or an adult section, either way is good), since a review written by a kid, will seldom show the level of detail or depth, as a review written by an adult, and kids should not be punished for this, they are only kids, and they might not see lego for much other than a fun toy. :)

I haven't checked to see if any of my reviews are gone, but as many of you, i often spend a great deal of time when writing a review, so i would not be happy if they disapper. I can only trust that people are honest enough, so that a review only goes down, if something is wrong with it.

A comment section en the reviews might also be a good idea, so osther users can correct if there are errors, or comment on why they might have ratet the review bad, or good as the case may be. :)

Posted by Huw in United Kingdom, 27 Oct 2010 14:40

@Krogh, thanks for taking the time to do so...

Posted by finger2314 in United Kingdom, 27 Oct 2010 14:49

I looked at my reviews to see if any were gone, and I saw that the helpful/ unhelpful box was there for me to use. Can you rate your own reviews or is that accidental? Would that not be unfair?

Posted by Sky-guardian in United States, 27 Oct 2010 15:33

Great idea!! However I have noticed that when looking at your own reviews you can rate them.... Might want to fix that otherwise people can boost their ratings.

Posted by Huw in United Kingdom, 27 Oct 2010 17:13

Yes you can rate your own reviews. I guess I could stop it but it hasn't been a problem before.

Posted by Town in United States, 27 Oct 2010 20:54

While I do like the idea of ordering the reviews by their 'helpful' rating, I also do like to read new reviews. Especially if the set was released a while ago, and the initial excitement of having-the-latest-toy-and-reviewing-it has worn off.

Maybe just a link at the top of the reviews; 'Read latest reviews'.

Posted by sudorum in United States, 27 Oct 2010 21:56

What I would suggest doing is implementing a comment box so the individual rating the review can say why they found the review to be helpful or unhelpful. My reviews of the Tantive IV Rebel Blockade Runner and the Burrow are two specific examples of reviews that for some particular reason are viewed as unhelpful by the majority of people. What bothers me, is that I really can't decipher why they were viewed so negatively.

Posted by Camo Hat in United States, 27 Oct 2010 22:55

a lot of the time you cant really tell how good a review is based on the ratings because not many people rate reviews. it would be easier to tell the good from the bad if we had a proffesionals opinion. im sure the administraters rate a lot of reviews. how about when an administrater marks a review as good or bad it says so. that way people wouldnt waste time reading unhelpful reviews. i also like the idea of a place to comment on reviews. also , it would be nice if we could review minifigs and comment on things like "this piece is made of rubber","this isnt worth getting",or maybe"dumbledore has a ladies dress on under his beard". i wouldnt have bought that magnet set if i knew that.

Posted by CharlieTheFunEngine in Australia, 28 Oct 2010 02:56

@ Dr Leg O' Brick - Finally someone agrees with me about Fantasy Era. Terrible, terrible stuff. I read your XL Grey Baseplate Review before it was taken down. Funny as Indy with Yoda's head. Sigh...

EDIT: Okay, now I'm confused. My Fantasy Era review is back and it says "Only 1 out of 7 people thought this review was helpful so has been rated as unhelpful.", but it's there agian. I'm happy and confused at the same time. Hapfused.

Posted by Doctor Leg O'Brick in United Kingdom, 28 Oct 2010 03:34

@TinyTurbosCityAndToyStory, well what would i know, my baseplates back! "3 out of 8 people thought this review was helpful" ha, i need to make my reviews better.......and yes, a burst of hapfused is coming......and fantasy era was all right, but not all the sets are good.....

Posted by Duq in Ireland, 28 Oct 2010 03:39

From my own experience a critical review gets more 'unhelpful' votes than a positive one. Maybe there's a way to have two votes:
1) Was this review helpful?
2) Do you agree with the verdict?
Not sure how this would work, it's just a suggestion...

Posted by Corellian Engineering in Belgium, 28 Oct 2010 05:26

I see the advantages of filtering out the review database, and using an algorithm is maybe the only way to do it.
(Unles the moderators take a couple of weeks off from school/work and dedicate themselves fulltime to this..)
But, I dont think its a good idea that the system itself hides/erases then automatically the "unhelpfull" reviews.

Maybe the system should, when a review gets a 2out of 5 (or whatever the key is), send a message to moderators who then can decide if its justified or not.
In the beginning this still might be a big job, but at least not all reviews need to be verified and after a while I guess it will be easy to keep up with.
The big advantage is: this way you avoid usefull reviews being cancelled by a "blind" system.

A good example: I was yesterday reading reviews for the first 1999 SW landspeeder ( http://www.brickset.com/reviews/?set=7110-1 ). One of the most elaborate reviews there gets just a 2/4 and is thus virtually in danger... while some of the simple pro/con lists still get the full 100%..

I noticed in many reviews the same as Duq.. critical reviews, talking a bit more in detail about pros and cons, about the actual qualities of the Lego set rather then about the popularity of the topic it represents, are sometimes receiving relatively much "not usefull" markings..
While a lot of the "This set is soooo coool, because X is my favourite theme/character/ship... that is why its the best set ever and realy awesome" are only marked usefull..
While the first catagory if reviews are in fact the ones that most interest me..

Seems to me that a lot of people mistake the (un)usefull rating as a way to (dis)agree..

Oh, yes, a last point: what with the reviews that dont get no rating at all? (because old / people dont read them / dont bother to rate).

Posted by Huw in United Kingdom, 28 Oct 2010 11:01

We could argue about this forever. Personally I think we should just see what happens now: time will tell if the new system works and if in a few months time I run a poll to see if the people's perception of review quality has changed we should find out.

I quite like Camo Hat's idea of having 'recommened reviews': that might be something that's not too hard to do as we moderate them.

Posted by vynsane in United States, 28 Oct 2010 13:27

I would say having a drop-off quotient for reviews on set pages is a very good idea. However, perhaps all reviews, regardless of rating, should still be available for viewing when one clicks on 'View all reviews posted by XXXX'.

Posted by Gus263 in Brazil, 28 Oct 2010 16:10

One thing that must be taken in consideration are the author's own way of rating a set. The rate is a crucial part when it comes to rate a review. For a children, maybe the most important part. Then, if a person don't comprehends the way the author uses to rate the sets, it can become unfair such system.
But in most of cases, I agree with it. Even though people spend less or more time writing reviews, some are needlessly complaints. For example when it comes to themes distant from the authors' view: If a guy just buy for example an action figure as "test" (supposing he only buys minifig-themes) and then breaks down the set in its review and rating, that'll be (at least for me) an unfair review.
In my profile, I said people could contact me if they had suggestion for me improving my reviews, and few contacted me. But now, it'd be pointless tell about reviews from more than a year ago...

Posted by Kalhiki in United States, 28 Oct 2010 22:38

I'm not sure what to think on this. I could write a big in depth review, but some people may not find it helpful, thus causing it being taken down. All that work gone to waste. However, on the upside, it will make people put more effort into their reviews rather than just being their review as "This is a good set, get it." It'll also probably increase my review quality, because I tend to write lengthy reviews, and I don't want to go through all that hard work just to have it taken down.

Posted by vynsane in United States, 29 Oct 2010 00:37

Kalhiki, I can only speak for myself, but I don't mark well thought-out and thorough reviews as 'unhelpful' regardless of whether or not the review actually 'helped' me in some way - I guess I use it more as a "was this review worth reading" rating, rather than "was this review helpful". Somewhat thorough reviews might get marked as 'helpful' but usually I just leave them alone. I only really mark reviews 'unhelpful' if they're a waste of time to read.

Posted by CharlieTheFunEngine in Australia, 29 Oct 2010 03:12

@ Doctor Leg O'Brick - Congratulations on getting your baseplate review back, but shouldn't my skeleton and your baseplate still be hidden?

@ vynsane - I don't think any reviews are "helpful" by definition, unless they tell of a new use for a part that could be used for MOCs or something similar. Whether a review is "helpful" or not is probably not the best rating system, and something like "was this review worth reading", as you put it, could be put to better use.

Posted by vynsane in United States, 29 Oct 2010 12:37

I'd venture there are many reasons a review *could* be 'helpful', be it that a positive review pushes someone who might be on the fence about a particular set into buying it, or that the parts are useful or rare, etc. Being helpful is a good quality, but perhaps that's not the best system upon which to rank reviews. Maybe reviews could have their *own* 5-brick rating system, which would allow for more gradation of the quality of the review, beyond the 'pass/fail' system of 'helpful/unhelpful' we have now... Just a thought. I realize it could drastically over-complicate things, or be a bit confusing in regards to the in-review 5-brick ratings, but it might be something to think about.

Posted by Sky-guardian in United States, 29 Oct 2010 14:19

I think helpful means more "did you find out what you wanted about this set?" or "did the review describe "the set?"
@vynsane i agree with your statement "was it worth reading." Helpful can be used too many ways....

Posted by Brickmatic in United States, 29 Oct 2010 19:57

I think the problem we are all running into is that people are looking for different things when they read a review. Some want a quick pro/con breakdown, some want a detailed comprehensive assessment, some want specific questions about a set answered, etc. At the end of the day, what is helpful to one person is unhelpful to another. Some people have picked up on this with the distinction being made for a a kid would look for versus what an adult would look for in a set. However, I think even amongst your own age group, whatever that may be, people are looking for different things. For this reason, I don't think any reviews should be hidden. I think it is better to allow the reviews to be sorted by popularity. Instead of a long question about helpfulness, I think a simple like button would suffice. The more likes, the higher up on the list the item will go based on an algorithm that also takes newness into account. The newer the review the higher it goes as well.

Speaking of like buttons, I think that a like / neutral / dislike button for each set would go a long way to get more people to rate sets. It wouldn't replace the reviews, but complement them. I know I would rate every set I bought if it were easy as clicking a button once. I suspect if there were such a mechanism, you'd get a rating from a much broader set of people than with the current system. And you'd still have reviews for people that enjoy adding more detailed impression of the sets.

And lastly, another feature idea that I think would be beneficial is to allow people to add reviewers to a favorites list. Then you could add the option to filter displayed reviews to show only reviews by people on your list. This would reward reviews who right interesting and valued reviews. For those looking at reviews, it would allow a way to quickly display reviews from people that have a style of reviews they like. For those writing reviews, it would reward them with the sense that there are people who really like their reviews.

Posted by Gus263 in Brazil, 30 Oct 2010 13:59

@Brickmatic - "At the end of the day, what is helpful to one person is unhelpful to another." "However, I think even amongst your own age group, whatever that may be, people are looking for different things." - I agree.

Commenting has ended on this article.