What is an ISO 12233 Chart?
The ISO 12233 Chart is the I3A/ISO standard for measuring the resolution of electronic still imaging cameras. The chart I am using is a very expensive, finely printed variation of the ISO 12233 standard that extends resolution to 4,000 lines per picture height (l/ph) and features other benefits as well. This chart serves as a good visual indicator of sharpness as well as CA (Chromatic Aberration) and distortion.
About the Test Shot Setup
If the test shots are not perfectly executed, the results are meaningless – or worse – deceptive. I go to great pains to make sure the test images are accurate:
The camera is mounted on a large, solid Manfrotto 400 Deluxe Geared Head which is mounted on a 158 lb (71.67 kg) Foba DSS-Gamma Studio Camera Stand or a 145 lb (65.8 kg) Manfrotto 809 Salon 230 Camera Stand. The camera stand is positioned on a custom designed, perfectly level, specially painted, 6" (152mm) thick concrete floor. The camera/lens is multiple-laser-aligned to the target which is mounted on 60" (1524mm) square, 1/2" thick glass (focal lengths over 460mm may be tested on a smaller-sized target). Focus assist lighting is from two Solux continuous lights. Test shot lighting is provided by four Canon 580ex II Speedlites or by four Nikon SB-900 Speedlights. I've tried many other lighting solutions, but this is the one that provides clearly the best results.
Tests are conducted using computer-aided Live View manual focusing (with center-point-only autofocus additionally tested). The best of the many re-focused shots (typically at least 10 sets and often 15-20) are used for the results for each camera/lens/focal-length/aperture combination. The proper lens hood is in place for all test shots and all filters are removed. Image Stabilization, if available, is turned off.
About the Shot Settings
All test shots are taken in RAW format using Canon or Nikon's Neutral picture style (all parameters = 0). All aberration corrections are turned off both in-camera and during post processing. Using DPP (Canon's Digital Photo Pro), a sharpening setting of 1 is added to Canon shots. Nikon images are processed in Nikon ViewNX with a sharpness setting of 0.
The sharpness parameter makes a big difference in the results. I chose a setting of 1 for Canon images because it is the minimum setting I use in my real world Canon shots. Nikon reviews came later and the setting of "0" was the closest match I could find to the Canon settings. View the comparison of the Canon EF 200mm f/2.0L IS USM Lens and Nikon 200mm f/2G IF-ED AF-S VR Nikkor Lens to see how two of the best lenses available compare across the brands. I see slightly more sharpening halos in the Nikon results, but the two sets of images are very close aside from the resolution differences in the cameras. Especially close considering that two completely different image processing pipelines are being used by these manufacturers.
As I said, the sharpness parameter makes a big difference in the results, and very weak sharpening is being used in the results shown by this tool. Keep this in mind when viewing the results. Contrast, saturation and all other parameters are left at flat 0 / neutral / low values. Pictures are converted into 16-bit TIFF files for processing.
Why Were These Crop Sections Selected
The top 100% crop (1 on the chart above) shows a pattern that is slightly to the right of center – for evaluation of the center performance of the lens (typically the sweet spot for a lens). The middle 100% crop (2 above) is taken from the position that, when shot on a full frame body, represents a corner on a APS-C (1.5x or 1.6x FOVCF) camera body. The bottom 100% crop (3 above) is from an area near the top-right corner of the image – regardless of which FOVCF body took the shot. Keep in mind that your subject will very frequently not be in the center of the frame (many use the "Rule of Thirds" for their composition).
Why Do the Pattern Sizes Vary
While I go to great lengths to get perfect test shots, there may be very slight variations in the framing of the tests (usually not more than a few pixels). I do not think these variances are enough to sway any comparisons – otherwise I reshoot the test. Still, some graphics in the test crops vary in size. What you are probably seeing in this case is lens distortion.
How Can I Evaluate for Lens Distortion?
To compare lens distortion using this tool, I like to choose a lens in the right selection box that is nearly distortion free. For Canon, the Canon EF 200mm f/2L IS USM Lens is a great choice – I have results for many cameras included for this lens. The Canon EF 200mm f/2.8L II USM Lens is another good choice. For Nikon, select the near-distortion-free Nikon 200mm f/2G IF-ED AF-S VR Nikkor Lens.
Then, choose a lens and focal length in the left selection box that you want to test for distortion. Make sure the same camera is selected for both samples. Watch the top-most crop as you switch the displayed results (using a button click or mouseover) between the compared lenses. If the reviewed lens shows larger details than the distortion-free comparable lens, it has barrel distortion. If the reviewed lens shows smaller details than the distortion-free comparable lens, it has pincushion distortion. The difference is the size of the details indicates the amount of distortion present.
How Can I Evaluate for CA (Chromatic Aberration)?
This chart is black and white only. If you see other colors – most often in the corners (bottom two crops), the lens exhibits CA.
Why Does The Bottom Crop Sample Vary in Brightness?
Variances of brightness in the bottom crop sample can be attributed to two things. First, at wide apertures, the difference is usually light fall-off/vignetting. This difference is very helpful to see and compare. At narrow apertures, where light fall-off is no longer an issue, the power of the flashes must be turned up dramatically. It is very hard to illuminate the chart perfectly evenly at this flash power level and the corners may be slightly brighter due to light fall-off across the chart from the flashes. Also, even though the lights are close to the test target plane, reflections are still a problem when testing super-wide angle lenses. I've implemented a newly-available larger test chart in 2008 to help with this issue.
Are The Crop Samples Accurate Image Quality Indicators?
For the most part, I find the results to be very indicative of the image quality of the lens being tested and the comparisons work very well – the results are even better than I had hoped for. However, I'm not totally satisfied with the Canon EF-S 17-55mm f/2.8 IS USM Lens results – the 17-55 under-performed in the comparison tool images in my opinion. And the cheap Canon EF-S 18-55mm f/3.5-5.6 IS USM Lens performed better at the wider focal lengths at the relatively short distance than I would expect to see from this lens – it over-performed in the comparison tool.
How To Get the Most Out of the Comparison
Any lens/camera/focal-length/aperture/extender combination tested can be compared to any other. Select the first test on the left and the comparison test on the right. You will see the test results selected on the left until you hover your mouse over the result images (or the arrow) – then the compare-to test results appear. Moving the mouse back and forth toggles the results – my favorite way to visually compare lens and camera test results. You can even compare a lens to itself. Additionally, the arrow buttons between the lens models can be clicked to lock the displayed results (ideal for mobile device users).
Lenses should be most-critically compared to each other only with test samples from the same camera body as it is the combination that is tested. Cameras can be critically compared to each other using the same lens (the Canon EF 200mm f/2L IS USM Lens and Canon EF 200mm f/2.8L II USM Lens have results from many cameras included).
Remember, looking at 100% crops from these high resolution cameras is similar to viewing a huge enlargement from a close distance. This chart is tough on a lens – and the selected Picture Style, sharpening and contrast levels shown are very low. You should expect real world images to look at least as good – and likely better.
Share the Comparisons
The comparison pages are coded to allow saving and sharing a link to a specific comparison. This feature should be especially helpful to the many valuable forums discussing such information.
Some Final Words ...
Obviously, I have invested a huge amount of time and money in this project. It is my hope that I can contribute to the photography community by making camera and lens selection easier. I always appreciate suggestions and comments.
Back to the Lens Comparisons.