DXOMark sensor rankings: Nikon owns seven of top ten positions for DSLRs!
Before I start, I just wanted to make it clear that I understand this article is going to be many things to many people. To some, it is going to be a discussion of the latest trend in DSLR sensor technology. To others, it is “obviously” a study in media / industry bias, and nothing more.
Anyways, DXOMark just published their test of the Canon 1DX sensor. It has indeed taken the top ranking spot for any Canon DSLR sensor, however it is just a single point ahead of the Canon 5D mk3 sensor, which according to DXOMark’s testing info is 1/3 of 1/3 of a stop difference. And it is a whole 14 points behind the champion Nikon D800/E…
Thus, we come to the statement that would start an all-out “flame war” on most DSLR community forums: Out of the top ten positions for DSLR sensor rankings, Nikon owns seven spots, Pentax owns one spot, and Canon owns two. And Canon’s two spots aren’t even close to the top either, they’re #7 (the new 1DX ranking) …and #10. (The 5D mk3)
(Note: A few medium-format sensors have been filtered out)
Let’s discuss the three main meanings / messages that most people will gather from this kind of information…
1.) The rise and fall of “king of the hill” companies, AKA the leap-frog effect.
Technology is always progressing. And there are always clear leaders when it comes to innovation and performance. 5-7 years ago, before Nikon started creating full-frame camera bodies, Canon reigned supreme. Actually, even among crop-sensor camera bodies Canon had far superior sensor technology when it came to low-light high-ISO performance. The Canon 10D and 20D alone made a laughing stock out of pretty much any Nikon crop-sensor in production. I remember, because I had a Nikon D70 and it was downright depressing to compare my ISO 1600 images to anything from Canon.
But now Nikon is on top. (Allegedly, but we’ll get into that in a minute) Full-frame cameras and crop-sensor cameras alike. DXOmark categorizes their results into color depth, dynamic range, and ISO perfromance. In the dynamic range category the results are especially surprising; Canon’s absolute latest technology does not even break into the top forty, in fact #40 is Canon’s first spot on the chart: the 1Ds mk3. The 5D mk3 and the 1DX are actually a step backward in dynamic range. They perform best in the ISO ranking, but are still 6th and 8th behind the Nikon D3S, D600, D4, D800/E. The 2008 Nikon D700 is even a few points ahead of the 2012 Canon 5D Mk3.
The question is, will Canon get back on top? Is this just a cycle, a phase? Personally I believe so. The under-dog always tries harder. Canon will return with a vengeance within a generation or so, you can bet.
2.) The next question is of course, …are these types of results even believable? Or could they all be biased or even faked?
You know what, I bet to some extent DXO Mark has been rooting for Nikon as the under-dog, and now is continuing to “put Canon in it’s place”. However I do have a habit of just being suspicious for no reason, or seeing what I want to see. For example one thing I noticed is that the Nikon D3200 (a $700 crop sensor DSLR) and the Canon 5D Mk3 (a $3500 full-frame DSLR) received the same overall score of 81, yet the D3200 was placed above the 5D mk3 on the chart. Even though the first two categories are extremely close, and the last category (ISO) …the 5D mk3 is over DOUBLE the score of the D3200. Clearly, the overall rank number should not be taken at face value!!! I never take anything at face value though. But, bold-faced lies? Are we being duped just because their charts look all high-tech, and they act like they’re some clean-room laboratory that couldn’t possibly measure wrong, or publish misleading results? Personally, I hardly doubt there is some grand conspiracy at play here. I think it is safe to trust the rankings. But we’re still forgetting one thing…
3.) Truth or not, the lab is still the lab. The final question is therefore: What does this mean in the real-world?
In the real world, are Nikon and Canon sensors really that different? Thankfully, the answer to that question is a resounding NO for almost every type of photographer, even high-end pros. Especially in the high ISO and color depth categories- in real world use you will barely notice any difference whatsoever among the top ten cameras, probably even among the top twenty. 90% of photographers out there can safely choose a camera system based on everything OTHER than sensor performance. Important things like lens selection and performance, or body control layout and customization, features, etc.
You’ll have to really, really push the envelope to start noticing the differences. I’m not interested in talking to armchair-envelope-pushers, either. However if it’s your job to shoot in the absolute worst possible light, all day long, then yeah you might want to consider getting a Nikon D3s. Or if you’re a renowned landscape photographer in search of absolutely mind-boggling dynamic range, then give the Nikon D800 or our recently reviewed Nikon D600 a try. And while we’re on the subject; if you’re serious about HDR landscapes then you may want to think twice about a Canon 5D Mk3; even the older Canon 5D mk2 is a better choice for digging deep into extreme shadow details…
Remember that I said 90% of photographers out there can safely choose a camera for other reasons. Yes, even you armchair-pixel-peepers who make a huge deal out of test results and rankings… When was the last time you actually pushed your camera so close to the envelope edge that these things became a show-stopping issue, in the real world? Or have you been too busy taking low-light sample photos of your pet, kid, or some boring still-life setup? For every sunrise that the average “self-proclaimed hardcore landscape photographer” has photographed, I bet they’ve spent five hours online debating the possible results of those images, and how they might just switch because they can’t live with their current setup anymore. For every basketball game or gymnastics competition that the average low-light shooter has photographed, I be they’ve spent ten hours online debating “signal-to-noise ratio” and the differences between chroma and luminance noise.
My advice? Get out there and shoot. Sure, it’s fun to nerd-out from time to time when a new camera is announced or tested. And yeah, if you’re extremely specialized in one area of photography, and if you have legitimate reasons to push the envelope way past it’s limit, then sensor performance is one consideration. Even then it’s still just one consideration of many, many others. Most of which are a lot more important than numbers on a chart.