Mechanische camera's

On lens reviews

 
Historical background
 The lens in front of the camera is the most important component of the imaging chain. The characteristics of the lens define what the detector (film or sensor) can capture (register) of the scene in front of the lens. Ultimately however, it is the eye and brain of the viewer of the photograph who evaluates the picture, irrespective of the intrinsic qualities of the optics involved. Seeing/observing and interpreting are two different acts and here we encounter the fine dividing line between art and science, that is especially important in photographs, being the result of a mainly mechanical process of recording part of reality.
The invention of photography was initiated by the desire to create the ultimate mechanical device for objectively (untouched by fallibilities of the limited human visual apparatus) recording reality.
The techniques of naturalistic representation were cultivated from the times of Leonardo da Vinci. The laws of the paths of geometrical light rays were also known then and when we assume that these rays when projected on a plane surface will depict nature in an accurate recording, we also accept that optical and mechanical tools can handle this without human interference. 
Daguerre and Talbot invented two different methods of mechanical image-making and from then on we are in a discussion about the truth and accuracy of the picture: from a picture does not lie to a photoshopped picture is a lie we can cover the whole range of values and interpretations. 
Whatever the position on the accuracy of the representation by optical-mechanical means, it is a fact that the picture is not invariant to human influences: a picture is basically defined by the choice of viewpoint, and arrangement and lighting of the subject: any landscape or portrait photographer knows this. 

And when the picture is taken and fixed and then presented as a photographic artefact its ultimate impression is dependent on the conditions of its viewing, not to mention the psychological condition of the viewer., when viewing and interpreting the picture. 
The complete chain from taking a picture to viewing that picture by an observer is a most complex one that by my knowledge has not been given proper attention. 
The chain can be broken down to two different parts: the image taking chain and the mage viewing chain. The first part can be assessed in a more or less scientific way (as long as we restrict ourselves to the optical-mechanical part), but the second part is almost immune to analysis. The pupil, lens and the detector inside the camera can be scrutinized and controlled, but these technical parameters cannot dictate how the human visual system interprets the picture within its own set of parameters of sight.
The optical system (lens) is one the main elements that define what we ultimately see and perceive in a picture. It is then natural that many if not most photographers want to know the characteristics of the lens they are about to buy or use.
Two approaches are possible.
The Mike Johnston approach.
 The first approach takes the whole imaging and viewing chain as an integral process and reviews a lens for its impact on the presentation of the scene as fixed on a print and viewed by an observer. Ihis is a most valid approach, but the quality of the review depends on the individual qualities of the observer.
This approach is best represented by the reviews done by Mike Johnston, whose lens reviews are jewels of impressionistic analysis. These commentaries are very good as long as they are seen as intelligent impressions about the behaviour of a lens when set in the context of the act of viewing the print by the photographer or any viewer.
These reviews are not intended to be lens tests in the strict sense of the word: A lens can be separated form the imaging and viewing chain and studied for its properties of accurately recording arrays of light rays emanating from the scene. Then we can study the lens as is and the MTF graphs are currently the best tool to assess the intrinsic performance of the lens. Whether these qualities will surface in the print depends on the expertise and knowledge of the photographer.
The Geoffrey Crawley approach.
 A lens test by an optical expert will produce a series of quantified measurements of lens parameters (transmission values, contrast values, aberration curves etc), that are in need of interpretation for photographic purposes. This is the approach of the famous lens tests of the British Journal of Photography during the period that Geoffrey Crawley was the editor and chief tester.  The mix of optical tests and practical evaluation backed up by a solid knowledge of the interaction between optical and photographic characteristics constitutes the best basis for a representative test result.
A third way
 The internet and the ubiquitous use of the digital camera and the computer screen as the main tool of presenting images produce a third type of lens review: it is now very easy to take a few pictures, compare them side by side, note a range of differences and comment on these differences. There are countless websites where you can find this kind of review. The main problem with this approach is the impossibility to separate the whole from the individual parts and facts from subjective interpretation. When you view and interpret an image file on a computer screen, you see the sum of the effects of the scene itself, the lens projection on the detector, the characteristics of the detector, the characteristics of the post processing program, the characteristics of the computer screen itself and last but not least the interpretative value system of the viewer.  To extract from that total system the relevant properties of the lens as invariant from the whole context of the imaging and viewing chain, is an impossible act. The final review may present interesting reading, but it is far removed from the two approaches indicated above. The value of the review is too much dependent on the personal qualities of the reviewer and when I want to read personal viewpoints about lenses, I prefer Mike Johnston.
The case of the 75mm lenses
 A recent review of the Leica and Cosina 75mm lenses may illustrate these points. Both lenses have been extensively tested and reviewed all over the internet and in many printed publications, and MTF graphs are available too. One may ask what is the added value of the constant repetition of known facts. In this context I am always reminded of one of the classical stories of Donald Duck: Duck is asking money from a rich foundation to discover America. The obvious reply is that America has already been discovered, and the answer by Duck is: yes, but not yet by me.  
The general conclusion from the review: both lenses perform quite well and are of comparable quality is nothing new and the more impressionistic comments depend heavily on the particular choices in the imaging and viewing chain.
I have on my desk the measured results of the two lenses, Leica and Cosina.
Here we have facts that have been collected for the lenses without any additional components from the imaging chain and are representative for the intrinsic quality of the lenses. The field of view corresponds to the normal 24x36mm frame and is not restricted to the smaller angle of view when one uses results that relate to the smaller M8 or Epson sensor. These results neglect the important outer parts of the capture area on the negative area as used in film-loading cameras like the M6 or M7 or MP. Comparing the MTF graphs we note that in the centre of the image and wide open both lenses perform equally well, but when looking at the outer zones there is a significant difference in correction philosophy. Leica has opted for a higher contrast, but allows more curvature of field and more astigmatism, where Cosina goes for lower contrast and less astigmatism. This correction philosophy has impact on the out of focus (bo keh) character of the lens.  It is also intriguing to see that the Cosina designer has opted for a higher contrast in the sagittal direction, where the Leica designer has selected the tangential direction to be of higher contrast. The main differences between both lenses in this area of definition can be  found in the 40 lp/mm of resolution. Most photographers have difficulties to find scenes where this level of definition can be captured. In itself this is interesting, but one needs to know that the higher resolutions are not only needed for the definition of fine detail, but also for better edge contrast at the lower frequencies.
The bigger differences between the two lenses are in the area of chromatic correction, where the Leica can show a much better apo-like correction. The performance of the Leica in the wavelength range of 400 to 500 micron is much better (by a factor of three). The upshot is that pictures taken with the Leica lens in scenes with a dominant blue light have a significantly higher contrast. 
The illumination of the full image circle (radius 22mm) is quite even in the case of the Leica lens and worse in the case of the Cosina lens. You do not only note that in the amount of vignetting at the corners: the Leica at 2.8 is much better than the Cosina at 2.8 (difference is more than one stop).  The illumination differences between Leica and Cosina at full aperture do present themselves in a hot spot in the centre of the image for the Cosina where the Leica has not such a spot. Transmission measurements also indicate that the Cosina is in effect closer to a true full aperture of 2.8 and not 2.5.
On a more practical level we may say that the actual comparison between both lenses is between a 2/75 and a 2.8/75. Then the performance advantages of the Leica lens are more impressive as a 2.8 lens is much simpler to correct to a high level of performance.
The Cosina  lens exhibits a higher level of focus shift than the Leica lens. The focus shift for the Cosina lens is within the 0.03mm CoC tolerance area, but when one wants to have high magnification prints, the focus shift could become visible.
Upshot
 At an aperture of f/4 both lenses perform visually equally well and that is what most reviewers will conclude. But such a conclusion can be made for most high quality lenses, especially in the realm of the M-bayonet lenses: lens design here has a long history and with modern design tools one can create good designs without much trouble. The step from a good to a superb design is not so easy, but then the naked eye might not be the best tool to detect the fine differences. The eye as an instrument for measurement is notoriously weak and one needs additional equipment to supplement and correct the conclusions of the eye. There is a dividing line between the seen and unseen and the eye is not good at detecting things one is not aware of. That is the basic but important conclusion of lens comparisons by the eye only: you see what you want to see and what you want to see is also influenced by the way one has configured the imaging and viewing chain. 
You cannot rely on MTF measurements alone, and certainly not on one figure merit values (this lens has a score of 78.3 and that lens has a score of 80.1), but also not on personal impressions. 
Lens testing such that reliable and representative conclusions can be drawn, is a complicated and time-consuming process. Without sophisticated equipment to back up the visual assessment of printed pictures (not screen analysis) lens reviews are of limited value when one wants to get a valid assessment of the intrinsic lens qualities.