Cultural Wedding Photography Guides (Launch DIscount!)

Gear & Apps

Don’t Be Fooled | 8 Signs Betraying A Gear Review That Isn’t Objective

By Lauchlan Toal on December 18th 2015

Gear reviews are a fantastic resource, and you can be sure that for whatever new lens or camera you’re planning to buy, there will be tons of reviews from across the world. While it’s great to have such a selection of resources, it can also be misleading.

Every reviewer is basing their judgement upon different criteria – the landscape photographer evaluates a lens differently than the sports photographer, just as the amateur photographer’s values differ from the professional photographer’s. If your needs and experience are different from the reviewer’s, you may be swayed by an opinion that really doesn’t apply to you – causing you to either buy gear that doesn’t suit you or to pass up on buying something that really would be helpful for you.

26-overall-score-1-stars

 

So how can you tell if the information in a review can be taken at face value or not? Knowing this will allow you to make better decisions based on what you’ve read online, and is also important if you write any reviews yourself – you want them to help your readers as much as possible. Here are 8 signs that a review isn’t completely objective.

1. The Reviewer Doesn’t Quantify Their Observations

I’m not saying that every review needs a half dozen Imatest graphs and a scoring system. Reviewers who quantify everything precisely are few and far between. But if the review states that, for example, AF speed is “fast” you really have no clue what that means. Someone who’s upgraded to a 50mm prime from their 18-55 kit lens may say that the 50mm lens focuses like lightning – because to them, it does. On the other hand, a professional sports photographer who’s used to the 400mm f2.8 might say that the 50mm prime has a fairly slow or average AF speed.

Without knowing the reviewer’s experience, it’s impossible to say if YOU’D find the lens slow or fast. Ideally, the reviewer would include some quantified observations like “the lens focuses from zero to infinity in 1.5 seconds on my Nikon D7100” to give you a more concrete value. However, if not, you can still do a little research into the author and try to figure out what equipment they’re coming from. This isn’t ideal, but it will help to give a little context to the review.

nikon-20mm-crop-sample

2. The Sample Photos Don’t Match The Text

This is a dead giveaway. If the reviewer says that a lens is really great, but the photos don’t reflect this, it’s pretty clear that their values don’t match up with yours. Sometimes they may also say that a lens is good at a certain type of photography, but provide no photos taken in that genre. Did they even test it for themselves? Be careful of these kinds of reviews, as the reviewer may just be trying to make money off you by posting great reviews of products they never even used, or by inflating their opinions of otherwise mediocre equipment.

3. There Are No Comparisons With Other Items

This goes with number one, in that reviewers can somewhat quantify their observations by comparing them to other pieces of equipment. For example, one could say that the 50mm lens focuses quickly, almost twice as fast as the 18-55. Then, if you have the 18-55 lens, you have some measurable idea of how fast the 50mm lens will focus. Alternatively, if you don’t have any of the gear being compared, you can at least look up other reviews for those items and see if they have quantified observations that can then be brought back to the comparison. For example, if you didn’t have the 18-55, but found a review stating that it focused in 2.5 seconds, you’d then be able to use that information to make sense of the comparison between the two lenses.

4. Observations Are Excessively Simplified

There is such a thing as too much simplicity, especially when it hides important information. Usually, this is seen more with the big reviewing sites, where they do a bevy of tests and then condense the results into a single metric. Some sites still let you access the original results, which is fantastic, but others just leave you with the bare bones. Other times, a reviewer will write a review which is little more than a conclusion, saying something to the effect of, “The gear is great, if you want better photos you should click my affiliate link and buy this now.” Both of these are of very little use, and shouldn’t be given much consideration in important financial decisions.

Sigma 12-24 DG II

 

5. The Reviewer Is Vague About Certain Details, But Not Others

This is an interesting case which could be attributed to simple forgetfulness, or it could be a lack of integrity. If the reviewer goes in-depth with certain metrics, but is vague on others, it’s possible that they’re trying to hide some flaws in the equipment. Alternatively, they may just lack the resources to test certain aspects fully. If all their reviews are vague about the same things, then that’s probably the case. However, if their reviews tend to vary in what they cover, I’d take their verdicts with a grain of salt.

6. Details Are Left Out

Just as bad as being vague about certain aspects is not talking about them at all. I could write a glowing review of the Nikon D7100 which would have every sports and wildlife photographer trying to buy one, I’d just have to leave out the little fact that the camera has a buffer of only 6 images. If you notice that a reviewer has left something out, there may be other things that they’ve forgotten.

nikon-d610-review-d800-d7100-d700-dslr-fashion-lifestyle-85mm-prime-nikkor-photography-9

 

7. The Review Was Written After Having Spent Very Little Time With The Equipment

Of course, this is fine if the reviewer states that the review is their first impressions. However, if they’re trying to say that they’ve had enough experience to write a full review after a three-day rental, chances are, they’re not going to be able to provide information that’s as accurate as someone who’s been using it for six months. There are important things like durability and workflow integration that you simply can’t judge without spending a lot of time with a new piece of kit, and hence, it’s impossible to create a full review from limited experience. This is why the best reviews always come out at least three months after a product goes on sale – the early reviews inevitably fail to mention certain crucial details.

[REWIND: Check out SLR Lounge’s extensive selection of reviews]

8. The Reviewer Doesn’t Mention Similar Alternatives

This is one that always bugs me a little bit – a reviewer’s goal is to help readers in the decision-making process, and if they don’t at least make a brief mention of similar products that the reader could look into, then they’re half way to being an advertisement. Reviewing the Sigma 50mm Art? Mention the Zeiss Otus, Nikon 58mm f1.4, and Canon 50mm L. Or at least one of them. If you’re reading a review and it seems like the reviewer has no knowledge of alternative products, it quite likely that this is the first time they’ve ever used this kind of equipment and their review may be thrown off by this.

By keeping these eight principles in mind, you’ll be able to tell when a reviewer is making a subjective evaluation that may not be of value to you, and avoid making the wrong decisions when it comes time to pull the trigger on new gear. Of course, the best way to see if you’ll like something is to try it for yourself! Whether you rent it, try it at the camera store, or borrow a friend’s stash, it’s always worth getting some hands on time with something you’re considering buying.

Any other tips for getting the most out of online reviews? Be sure to share in the comments below!

Lauchlan Toal is a food photographer in Halifax, Nova Scotia. When not playing with his dinner, he can be found chasing bugs, shooting sports, or otherwise having fun with photography. You can follow his work online, or hunt him down on the blogs and forums that he frequents.

19 Comments

Please or register to post a comment.

  1. Dave Haynie

    #9 The review is basically a paraphrase of the product’s marketing hand-outs.

    | |
  2. Barry Chapman

    So I see the url for this article says 8 signs betraying a subjective gear review, which is correct. So why does the title say 8 signs betraying an objective gear review?

    | |
    • Lauchlan Toal

      That is an excellent question Barry – I’ll check on that. Thanks for bringing it up!

      | |
  3. robert s

    today, I believe almost NO ONE regarding tests. everyone has an agenda and interest in you clicking their sponsored link to buy so they can make money.

    reviews seem to be “excellent” and “support our site by using our sponsored link”
    . shoot at the same composition and focal length at open aperture, 1 stop down and 3 stops down and ill decide. post UNTOUCHED raw files and let me assess. no need to be creative/artistic, I dont care for the persons skill. just shoot simple objects in a variety of situations. shoot the same frame 3 times for each focal length. open/1 stop down/ 3stops down. thats enough for me. and a tripod of course. if you shoot handheld, the pictures arent worth jack sh..

    | |
  4. Ralph Hightower

    Back in the old days when I got my first SLR camera, there were three variables, camera, lens, and film. I subscribed to Popular Photography because there was no Internet back then; it was just books (John Hedgecoe was a favorite) and magazines. Back then, Popular Photography would do camera, lens, and also film comparisons and reviews.

    I do love the autofocus of my 5D Mk III w/ the 24-105 f4L kit lens; but I have had to switch to manual focus on occasion. I also rented the EF 100-400 f4.5-5.6L Mk I & II and was impressed with the autofocus. But it is easy to impress me with autofocus.

    Thankfully, I do keep my focusing skills sharp by using my Canon A-1 and F-1N.

    | |
  5. Daniel Lee

    There are two things that usually annoy me about reviews I read (TDP are probably the best and least bias reviews)., these are:

    1. Reviewers who test lens A wide open against lens B one stop down to get the same aperture. Obviously lens B will have the advantage being stopped down yet they try to spin it like lens B is so much superior. If test one lens stopped then, then test the other in the same way.

    2. When reviewers get all surprised when a lens that is just released is sharper than one that was released 20 years ago. A food example of this is the ne Sigma lenses. Considering how new some of them are in comparison to Canon’s, I expect them to be better optically.

    | |
  6. Paul Nguyen

    Ultimately, I think the problem is that gear reviews should be subjective because using gear is a subjective experience. A lens and camera might be as objectively good as you want, but everyone has their own different pet peeves and things which annoy them.

    For example, just to show you what I mean – when buying M4/3 lenses, every review said that the Olympus 12-40mm f/2.8 was a better lens than the Panasonic 12-35mm f/2.8. However, after using both of them extensively (I actually went and bought both), I found that the Panasonic 12-35mm f/2.8 was a better lens because the zooming mechanism was smoother and less tactile, it was a bit lighter and I just found myself using it more often. Sure, the Olympus might have had better image quality, but my annoyances with it meant I didn’t really care about its better IQ.

    Another example is the Nikon 24-70mm f/2.8 vs. Tamron 24-70mm f/2.8. Regardless of what the IQ differences might be, the Nikon zoom ring is at the base of the lens and the Tamron’s is at the front – this is a source of genuine annoyance and something which I just find difficult to use (perhaps because I’m used to zoom rings being at the base).

    These small things might sound petty, but I think they’re the things that matter in the long run. They’re things which many reviews miss, but I try to incorporate into my reviews whenever possible.

    | |
    • Matthew Saville

      The Nikon 24-70 2.8 is known to be downright abysmal, as far as longevity / survivability are concerned, for the zoom barrel reason alone. It is very, very poorly designed and I have lost count of how many of my fellow Nikon users have had their 24-70 2.8 G jam up on them, or at least start grinding and sticking. Mine, after one small bump to the rear casing, started grinding so bad that aluminum flake dust started coming out of it.

      All in all, what I mean to say is, I’d certainly never buy the Nikon again, and would love to have the Tamron. Many folks these days are actually raised on kit lenses from the 2000’s which have always had the focus and zoom rings reversed, so it’s less and less of a deal.

      This isn’t meant to be argumentative or anything, it just goes to prove the original point: subjectivity and objectivity need to always be tempered with extensive testing, and even in some cases, research into common flaws or serious bugs / drawbacks that one might not be able to expose in an ordinary review.

      | |
    • Lauchlan Toal

      Absolutely, and I agree that your personal preferences are hugely important. And that’s why it’s crucial to notice when a reviewer’s preferences are different from your own, because gear really is so subjective.

      | |
  7. Sean Goebel

    Whoa, this is the best article SLRL has featured in recent memory. Good job.

    A few other things I would add:
    9) The reviewer doesn’t show 100% crops, or shows improperly-done 100% crops. Some people (even here) seem very unclear on how to produce a 100% crop. A 100% crop is an image displayed so that one pixel of the original image ends up as 1-pixel on the screen. IF RESCALING OCCURS, IT IS NO LONGER A 100% CROP. Images in SLRL articles are 750 pixels wide. Therefore an image should be cropped to 750 px (or smaller) and then included in the article at that same resolution. If the image is cropped to 1000 pixels wide, and then included in an SLRL article where it gets squished to 750 px, it is no longer a 100% crop.
    Similarly, trying to pass off a “100% crop” in a product review video is almost guaranteed for failure. In order for that to be done correctly, you would need to (for example) crop an image to 1) 1920 pixels wide; 2) set your video editing workflow to 1920×1080; 3) render the video to a 1920×1080 finished product; 3) get Youtube (or your video hosting platform) to support the video at 1920×1080 (Youtube seems to drop resolution options at random…); 4) the viewer must select 1920×1080 from the resolution options of the video; 5) the viewer must be using a 1920×1080 screen; and 6) the viewer must be watching the video in full-screen mode. If any one of these points are not followed, it’s no longer a 100% crop. This example holds true for other resolutions, as long as the numbers are consistent all the way through. This is why I cringe every time someone tries to show lens sharpness samples in a video.

    9b) If reviewing a lens, the reviewer doesn’t show crops from the corners AND at wide-open apertures. Anyone who as taken a basic-level optics class will tell you that a 1-element lens can produce flawlessly sharp images on-axis (i.e. in the center). Give me the cheapest, crappiest lens that you never use, and I will make it produce images that are beautifully sharp in the center. Any lens can do this. Showing an f/8 center crop is utterly meaningless. What sets lenses apart is the corners, particularly at wide-open apertures. Therefore, for a lens review to be helpful, it MUST show corners, ideally at a variety of apertures.

    | |
    • Paul Nguyen

      I don’t get the hype of corner sharpness. The solution would be to not put your subjects in the corner of frame. Beyond that, nobody apart from photographers look in the corner of frame anyway.

      | |
    • Sean Goebel

      Paul, corner sharpness matters in every genre of photography that isn’t boringly-composed narrow-DOF portraits.

      | |
    • Matthew Saville

      Paul, if you “don’t get the hype” of corner sharpness, that’s fine. In fact most common photographers really shouldn’t even bother worrying about corners. However there are definitely still plenty of applications in which off-center sharpness really comes into play. Personally, I just happen to frequently do two of the most demanding things a lens can be expected to deliver in this regard: I shoot multi-person group photos wide open at f/1.4 and f/1.8 on a regular basis, putting faces at and past the rule-of-thirds areas, and I also do astro-landscape photography where corner sharpness wide open is extremely important, as is vignetting, field curvature, and other things.

      Most folks really don’t need much more than a Toyota Corolla or Prius to get from point A to point B. But that hasn’t stopped the automobile industry from developing innumerable lines of cars that range from ten thousand dollars all the way up into the hundreds of thousands of dollars. Far be it from me to stop folks from buying something more high-performance than I personally need to commute to work! ;-)

      | |
    • Lauchlan Toal

      Thanks Sean, great points. Man, I totally forgot about “100% crops” in videos, but that’s definitely another thing which is pretty ridiculous. Glad you brought it up.

      | |
    • Paul Nguyen

      Thanks for the reply Matt, I’m a big fan of your work ever since I saw it. Given that, however, I know that you shoot a lot with the Rokinon 14mm f/2.8, which doesn’t have the hottest corner sharpness, so I think this whole notion that corner sharpness is extremely important isn’t quite true.

      In fact, one of my favourite lenses is the Nikon 17-35mm f/2.8, which is probably known for pretty bad off-centre sharpness, but that said, I’ve made great images with it and the only people who point out the corners are other photographers. There’s actually an interesting graphic once, might be from Canon, which shows how an artist’s eye and a layperson’s eye moves differently when looking at a picture.

      I agree with you 100% though, most people definitely don’t need more than a Toyota Corolla, and the same can be said about lenses. I guess I didn’t express myself well enough before. It’s not that corner sharpness isn’t important at all, it’s just that I think people spend a lot of time worrying about it over other important characteristics when it’s something that most people barely look at.

      | |
    • Matthew Saville

      Paul,

      In my tests and studies it looks quite the opposite, actually: the Rokinon 14mm has rather nice corners, even wide open; low coma and impressive sharpness, with the only downfall being vignetting which is something I don’t mind and/or can overcome. And that’s just wide open; by f/5.6-8 the Rokinon 14mm is as sharp or sharper than the name-brand lenses that cost 5-10X more. Yowza.

      Really, it just comes down to the intended purpose / usage of the lens. If I were shooting almost ANYTHING other than serious landscape photography and astro-landsdcape photography, I’d jump on that awesome Nikon 17-35mm f/2.8 instead, and never give a second thought to the extreme corners. Because really, for general photojournalism, all you need is a sharp area around and within the rule-of-thirds box. Extreme corners, those last bits that almost always get chopped off when you’re merely leveling your slightly crooked photos, just don’t matter.

      I do still push the envelope enough to be able to discern between the off-center sharpness of a lens like the Sigma 35 Art and its competition, even if extreme corners don’t matter. And I chose the Sigma because it is very sharp off-center, indeed nearly to the corners. And yes, I do put people’s faces out near the rule-of-thirds area quite often, wide open at f1/.4, something I’ve been previously disappointed with in other lenses.

      | |
  8. adam sanford

    Nice writeup. For the Canon camp, that’s why we read TDP. Bryan Carnathan’s fine work passes all eight of your criteria, Lauchlan.

    | |
    • Lauchlan Toal

      Thanks Adam – I agree, TDP publishes some fine reviews. Another great reviewer is Brad Hill of NaturalArtImages.com, who does some great work with Nikon wildlife equipment.

      | |
    • Lauchlan Toal

      Sorry, got the site URL wrong – it’s NaturalArt.ca – the site’s called Natural Art Images.

      | |