Home » Opinions » Digital Cameras » How We Tested The Cameras

How We Tested The Cameras

We use a number of tests to push digital cameras under a variety of conditions. For all tests, we set each camera to its highest optical resolution and best quality JPEG setting. The exposure and white balance were set to automatic.

To test overall resolution we use a PIMA chart which conforms to the ISO International Standard 12233, entitled “ISO Resolution Chart for Electronic Still Cameras”. This features numerous scales and wedges, allowing accurate measurements of lens and sensor resolving power. The chart is photographed at a point mid-way through the camera’s optical zoom range.
We concentrate on the horizontal and vertical resolution scales, labelled from 0 to 20. The results are in relation to the height of the actual picture itself, hence scores are measured in lines per picture height. The scale used on the chart should be multiplied by 100, hence 0 to 20 represents resolutions of 0 to 2000 lines per picture height.
We take a measurement from the point at which the lines can no longer be resolved separately. If this occurs at the position labelled 10, the camera system scores 1000 lines per picture height. The published images show a crop of the relevant scale reproduced at 1:1 for comparison.

To test optical geometry we photograph a rectangular grid chart at the widest and longest positions of the optical zoom; in each case, the target is positioned to fill the frame as measured using the camera’s electronic screen.
Geometric distortion is immediately apparent as lines either bending outwards (barrel) or inwards (pincushion). Generally speaking, there will be barrel distortion at the wide end and slight pincushion at the tele end. The overall view also reveals the screen coverage – a border around the target indicates the screen does not show the whole capture area. Our published result shows the entire frame captured.

To test for chromatic aberrations in the lens we photograph a black card with various cut-out shapes against a bright background, again at the widest and longest positions of the optical zoom. Chromatic aberrations are immediately apparent as coloured fringes around the edges of the cut-outs, particularly in the far corners; in our results we reproduce a crop of the top left corner. This is a controlled version of photographing high contrast compositions in real-life, such as the edge of a roof or tree branches against an overcast white sky. Ideally there’ll be no colour fringes, but virtually every lens suffers from them to a certain degree.

Our close-up test measures the smallest area a camera can sharply focus on in its macro mode. We do this by capturing the closest crop of a metal ruler as possible. By positioning the 100mm mark on the left side of the image we can measure the precise horizontal distance in mm which the camera can capture; the smaller the better.
Generally speaking the closest focussing distance is achieved with the lens at its widest setting, but some cameras can generate a superior close-up result with the lens at a mid-position and the camera held further back. Such results normally have less distortion and also allow for easier lighting of the subject. For each camera we will score its macro performance on the largest reproduction it can make of the ruler, whether at a wide or mid-zoom position. Our published result shows the entire frame captured.

When testing cameras we also take a great number of photos in typical outdoor and indoor environments. Since most of these are unrepeatable, we only use them to test general performance and illustrate specific points of interest. The exception is when we perform a group test, as the same real-life shot can be taken with each camera and used for comparison. In these cases, great care is taken to ensure each camera is capturing exactly the same view and for consistent lighting, all images are taken in quick succession.

The outdoor shot above was used to test and compare ten cameras during a September 2003 group test. In the results section for each camera we cropped three areas of interest and reproduced them at 1:1 for on-screen examination and comparison.

comments powered by Disqus