Leica M9, part 7, colour reproduction and noise revisited ( 2009)
In the days of silver halogen emulsions, grain, sharpness/resolution and tonal range were characteristics of the image that were intensely studied and measured. It was found that when grain became smaller, the sharpness impression decreased, the resolution increased and the tonal range decreased. Several other fixed relationships were established, like sharpness impression increased as grain increased and speed increased. Many photographers were disappointed to discover that finer grain did not bring greater visual impact.
Discussing graininess is quite difficult. A real comparison is only possible when lab conditions are strictly observed. One needs the identical density and the identical CI value (steepness of gradation curve) to make meaningful comparisons. If you compare films then preferably the same developer should be used. When comparing developers, the same film and same densities are required. This was seldom the case and all kinds of wild assertions were common. For decades the debate ranged between supporters that a specific film and developer combo would present best results. And claims about resolution ranged widely for let us say a Tmax 100 film between 50 lp/mm and 200 lp/mm. Notorious is the claim that a certain lens/film combo could resolve more than 400 lp/mm or the claim by the Gigabit manufacturer that this film could resolve around 500+ lp/mm.
The basic issue here is the fact that there are no reliable standards and definitions and that the number of parameters is so high that every claim can be made and even supported.
We have now unfortunately the same situation in the digital realm. In fact the situation is even worse. It is very easy to do a noise or resolution (lp/ph) or colour analysis: many programs are available and anyone can design his/her own analysis program. Understandably every result will be different. Not necessarily wrong. But it is impossible to directly compare results because of the greatly differing conditions. The only option here is a rational discourse, not a lawsuit. Comparing results and comparing methods is the best way to analyse and explain the differences. But camera users are quite often not interested in independent lab results, but in numerical values, however achieved at, that support their convictions and expectations. If you want to believe that the noise behaviour of the Leica M9 is better than that of the M8, you will not be inclined to read with interest and an open mind the results of a report that concludes otherwise.
This state of affairs has resulted in the establishment of international standards, most prominent the ISO standards. The ISO norms for, as example, noise and resolution define the procedures and test conditions that need to be used to analyze the topic defined.
Signal and noise are the prime characteristics for categorizing imaging performance. Signal is defined as any response that provides valued information and noise is defined as any response that detracts from a desired signal. A signal can be measured by two broad metrics: OECF (opto-electronic Conversion Factor) and SFR (Spatial Frequency Response). Noise can be measured by NPS (noise power spectrum) and spatial distortion, like aliasing.
For any metric there is a range of secondary measurements that define part of the metric. As example: OECF considers sensitivity, tone, exposure, white balance. SFR is related to MTF, but not the same.
A main characteristic of OECF is the fact that it is based on larger areas of the image (the region of interest). Noise analysis is often based on small areas of the image. Signal to noise ratio (SNR) combines both metrics, but there is a difference when calculating the SNR using large or small areas of interest.
In my test reports I use the programs that support the ISO standards, like Imatest, Image Engineering and Image Science Associates. Many of these programs rely on Matlab routines to do the job. Matlab itself has a wide range of image tools with which one can analyse and calculate almost everything in the digital imaging realm.
The results of the OECF measurement procedure are consistent with practical experience. The combination of dynamic range and SNR gives information about contrast (or signal) and the probability of detecting that signal, which is noise. You can measure signal as a separate density patch or as an incremental signal, like the classical Kodak step wedge. In the first case you use a noise-cracking technique that separates rms noise from random temporal rms noise. In the photographic practice the just noticeable density differences are often more important than the grain pattern itself. The calculation of the incremental signal/noise is the derivative of the OECF function.
I have expanded a bit on the theory behind the several types of measurements to warn against the indiscriminate adoption of results without insight into the way the results are arrived at.
Several image quality studies have found that SNR values of 40 and 10 represent excellent and acceptable levels of image quality. These levels are the basic for the OECF definition of good (that is useable) dynamic range.
The most important topic is not the fact that one can deliver numerical values, but the question how these numbers can be related to practical photographic topics. Just as grain is not always objectionable, so is noise. And the quest for the highest resolution is sometimes other-worldly. And there is always the topic that the screen does not represent the image as it is printed on paper.
It makes sense to re-examine the procedures when the results are being questioned. I photographed the Macbeth chart again with the M8 and M9, both fitted with the same lens. To keep the flow of data to a practical level I restricted myself to the ISO2500 setting. The chart was photographed with several WB settings, appropriate to the ambient lighting, in this case Auto, daylight and cloudy. The exposure was set according to the reading with the Gossen Mastersix and a range of exposures was made with half-stop increments over a range of +/- three stops. Both cameras gave an over exposure of one stop when the in-camera exposure meter (on Auto) was used, compared to the official grey-card reading.
This analysis points to several conclusions. The noise pattern is not consistent when using different over- and under exposure values. The pattern does change when using different colour calibrations (daylight, cloudy, flash) and the pattern does change when using different raw converters. None of these conclusions should come as a surprise. Leica users, still versed in AgX technique will recognize these trends as what you expect in grain patterns when using different exposures and chemical developers.
I checked the whole bunch of image files to find the optimum for the M9 and the M8 camera and then did the noise analysis on the bottom row of the Macbeth card. The results you see below, for the JPG version and the Raw/TIF version. There is only a slight difference between the two cameras and the M9 is a fraction better in the noise reduction. Note too that by optimizing the colour response, the blue cast of the M9 images is gone. Red the colour paragraphs for more information on this topic.
These are the optimum pictures. It became evident under the inspection of the large amount of pictures (I made more than a hundred pictures with every camera: grey cards with exposure bracketing and the full range of Kelvin values from 2000K to 12800K and the Macbeth card with the same options) that the M8 is a much more sensitive instrument than the M9 as far as optimum results are concerned. It is much easier with the M8 to take sub-optimum results than with the M9 which has a more stable result pattern. From this perspective the M9 is more user friendly camera that the M8. Even when you wander away from the optimum, the M9 brings excellent results. The M8 on the other hand brings much lower results when you do not operate at optimum parameters. This phenomenon may explain the fact that many M8/M9 comparisons favour the M9 results. The M8 can bring results as good as the M9 but needs to be operated within narrow limits. But if you care to move on from the role of Leica aficionado to Leica expert the M8 is a most pleasant and potent camera to use.
The overall conclusion that under many diverging situations and conditions the M9 delivers the more consistent quality is true too. The M8 operates in a more narrow band when optimum results are required and when these conditions do not apply the result is not as good as that what you get with the M9, at least at higher ISO values.
Colour profiles
The calibration of the colour response of the M8 and M9 do differ: when you study the Exif data, you will discover that the colour matrix has different values for both types of camera model. A direct comparison between the colour behaviour of the M8 and M9 is not advisable.
As example:Macbeth card, daylight illumination: the M9 on Auto sets the white balance to 4870K, Daylight to 4351K, Cloudy to 4996K. The M8 on Auto sets the white balance to 5509K, on daylight to 5083K and Cloudy to 5922K.
These values also change when using different ISO settings and when there is under- and overexposure. In itself this is not a problem. Experienced users will select the colour response according to taste and intended use. When selecting colour slide films you have the same problem. It is usual for critical work to use filters and to fine tune the colour reproduction. The recorded values were analysed using the new version 5 of Capture One. It will be no surprise to learn that other programs will give different values: as example the same picture in Capture One 5.0 gives 5102K, where PWP gives 8870K.
The colour calibration does also influence the noise pattern, especially in the darker parts of the scene.
Generally when using the RAW setting of the camera, the RAW program operates on the luminance values of the pixels and the Bayer pattern to construct the colour for every pixel. In most commercial programs these algorithms are proprietary and highly efficient. You cannot influence the algorithm, but you can sometimes choose between different ICC profiles. Several of the smaller RAW programs let you create your own camera profile. In all cases there are two profiles you have to consider: the input profile (the camera profile) and the output profile (like sRGB, Adobe RGB or wide gamut RGB). These choices have their effects on the final colour representation of the image. And none is perfect!
I used the Macbeth card and Capture One 5 version and the generic profiles for M8 and M9 to generate the image files. In this case I let the program decide what colour space to use and selected the Auto option. This file was then exported in the three colour spaces mentioned (sRGB, aRGB and wgRGB) and analysed with Imatest. The six results are shown below. Note that the differences overall are quite substantial and that within a colour space some patches are better or worse. It is very demanding to make general and sensible conclusions. One of the options is to do some calculations and find the average deltaE variations. This is a common practice, but I cannot recommend this one, as the deltaE variations assume a linear correspondence between numerical values and visual impressions which is not the case. The eye is much more sensitive to small variations in one colour than in another.
The second approach I used is an option within the PWP program. The Macbeth card has accurate values for every patch and these values are published. A program can read the photographed card and compare the values found with the internal accurate values and map accurate values on the photographed values. As values differ for every patch the program does a best guess to find the best overall match between recorded and accurate values. This I did for the M8 and M9 image and exported the results to the three colour spaces and analyzed the results with Imatest. Below are the results: they differ from the ones above, I would hesitate to add
The Macbeth card is a simple test as it only has 18 different colour patches and six monochrome ones. But even this limited range of colours is never fully accurately captured or calculated in post processing software.
But colour psychology insists that accurate reproduction of colour is not always the best option: if it were, the Kodachrome films would be still widely used. Here I can give only one advice: the advantage of digital capture and recording is the ease of use and the cheapness of the process. Experimenting with the several options (input profiles and export profiles and not to be overlooked printer profiles) for the type of pictures you like to make and see is the best approach. With film you are stuck with a limited colour bandwidth and a small possible overall change by employing filters. In the digital realm, there is a wider gamut of options. More like what painters have. Colour is primarily a psychological phenomenon relating to emotion and mood, not a colorimetric exercise. The Leica M8 and M9 do differ in this respect, but with careful calibration of the input and output parameters one can get the colours one likes and it is even possible to have the colour reproduction of both cameras to converge to a large extent. To claim that one camera is much better than the other is a dangerous game and an unnecessary exercise. And this goes too for the comparison between the Leica M9 and other non-Leica models, like the Canon or Nikon cameras. In straight recording Nikon and Canon have an advantage as their colour profiles are optimized for skin reproduction and other often selected colours. A point to consider is the fact that colour shifts in the M9 and M8 space have a more pronounced effect on the other colours where the high-end Nikon and Canon cameras have a more balanced behaviour.
To recap: colour reproduction is a science and a complicated one. It is very difficult, at least in my opinion, to make general statements about the colour quality of a specific camera. A reviewer can present as much factual evidence as one can handle, but subjective conclusions should be used very sparingly.
Where do we stand at this moment?
The Leica M9 is a most delightful camera to use and even own. It does not convey that peculiar proud of ownership one feels when handling a film loading M model, but as a photographic tool it delivers sound performance.
The characteristics of the M9 bring it close to the dslr competition without however blasting to the top. In this sense the M9 is the true successor of the M5, because that camera too left behind a few precious M features to add aspects that update the camera to the modern times.
It is still too early to label the M9 as one of the very best or one of the great or one of the many excellent ones on the market.
The current tendency to profile the M8 as a flawed toy and to profile the M9 as one of the best digital cameras in the world is a bit myopic and disregards the many qualities of the M8 as a photographic tool. The IR bias can be put to good use when doing black and white photography in the wider sense of the word. The M8 is a more demanding camera than the M9 is and to get excellent results with the M8 asks for some additional determination. But even the M9 is not a fully mature product and needs some expertise to take great pictures.
In the next part we will conclude this essay and offer some perspective on the role and position of the digital CRF in the current imaging world.