The basics
Any lens (excluding the enormously expensive lithographic lenses, only corrected for UV light) has aberrations. Aberrations are like the Russian dolls: correct the third order and the fifth order will pop up, correct all fifth order aberrations and seventh-order aberrations will emerge and so on. The task of the optical designer was and is the balancing of all these aberrations in order to fulfill the main goal of all optics: transfer the subject points to image points without any loss of information. A daunting task that every designer solves individually. Modern design is supported by software (in many cases the Code V program). Most software works with merit functions that are translated into numbers. The program then proposes an optical system (lens curvatures, thicknesses and distances between lens elements) when the number of lens elements is given. The optical designer has now the task to construct the merit function. The main and big problem is to find a merit function that can represent the practical requirements of the intended user.
This is indeed the primary problem: how to find a balance between aberrations that supports ‘good’ imagery, where ‘good’ has a considerable bandwidth of interpretation. Once fixed, the optical system has to be manufactured and the tolerances of the process define the final and actual performance.
Scientists at the Zeiss Company studied this problem quite intensively and discovered that the image contrast (global and local) was the decisive factor in appreciating the image quality when the limits of eye vision are taken into account. In is known for a long time that the optimal human eye can only resolve details that have a certain size. It has been established that the eye can resolve at most 10 line pairs per mm at a distance of 25 cm. Ctein claims that any person can distinguish between 20 and 30 lp/mm on a print, but that statement is certainly not supported by scientific or empirical evidence.
The problem
Reviews of lenses have the task to analyze the characteristics of a lens and to relate these characteristics to a lens profile, such that a prospective user can know what to expect of a lens. This requirement is more problematic that it sounds. There are basically two approaches to lens evaluation: (1) the scientific lab analysis and (2) the personal and subjective analysis of selected characteristics of the lens in question. I have to stress the element of ‘selected’, because any lens is a compromise between conflicting demands. When the reviewer is most interested in close up performance, because that is what interests the reviewer, all practical ‘tests’ and the importance of the conclusions will be focused on this aspect. When a reviewer is most interested in finding colour fringes, all his work will be focused on finding these colour defects. What these characteristics mean in practice or how important they are for the majority of users is not considered.
The current landscape
Analyzing the performance of lenses is still an important element of many blogs, review sites and magazines.
In the past, lens testing was a specialized occupation. Projection of patterns (lines, points), and MTF measurements were the classical ways of doing this. Zeiss used their own MTF equipment and most others (including Leitz) used projection patterns. Needed was considerable expertise to analyze these patterns. The important issue was always how to relate the results to practical requirements.
Since the digital turn in cameras the camera has become the virtual test lab. Software is often used to analyze the results on the sensor surface. Many magazines and some blogs/sites use the software of Image Engineering or of Imatest to present the findings. These findings are what the designers of the software find important. While often interpreted as tests of camera sensors or lens optics, the fact is that these tests always use both elements. Not considered is the problem of the hardware, the accuracy of the camera and lens, the influence of the software to translate the pixel photon count to visual images and so on. All measurements will have a tolerance level of let us say 5%. The often used parameter of the lines per image height is presented with semi-scientific precision and the camera/lens performance is ranked according to these values: when a system is measured with a value of 2856 l/ih and another one with a value of 2788 l/ih there is no discussion that the real values might be 2713 and 2928, in fact reversing the order. Even when the tolerance would be 1% there is no discussion of the real values, related to the measurement values.
It is questionable whether the numerical results can be directly related to the practical requirements, but one requirement is obeyed: the consistency of the tests and the consistency of the lab conditions.
This requirement is not guaranteed by the many subjective evaluations that ask for attention on the web.
These personalized and very subjective reviews fall in two extremes: the subjective reviews that are full of the words ‘impression’, ‘feeling’ and ‘opinion’. The choice of these words indicate the intention of the reviewer: not an objective analysis, but a description of the emotions when using a specific lens or camera body. A good example is the website of Joeri van der Kloet. A cursory glance at the discussion platform of the TheOnlinePhotographer will show you that almost every comment made there on whatever topic is at best simply an expression of personal preference and at worst the continuation of some myth. It is amazing what internet discussions can reveal about the human psyche!
The other extreme is the pseudo-scientific approach. A good example is the website of Mr Chambers. His recent review of the Leica Summilux-SL 1.4/50 mm ASPh is a fine example of this attitude. First of all, he does not describe the test conditions and uses a range of different pictures to support his statements (lab rule # 1is to state the conditions of measurement and to hold as many influencing parameters as possible constant ). Secondly he uses the method of pixel shifting to artificially enlarge the resolution of the sensor. Presumably this is done to improve the resolution yardstick for the optical performance. Martin Doppelbauer has a sensible explanation why this is a fake option. My own experiments with the Olympus PenF and the option of enhanced resolution are also not impressive: in fact the results are worse than the original ones! Thirdly he is obsessed with the phenomenon of the occurrence of distortion and the use of software to correct this so-called aberration.Again, without scientific evidence, there is the claim that this software correction will reduce the optical performance. Any modern book about optical design will tell you that distortion can be easily corrected by mathematics without loss of information.
The last issue I would like to address is the pseudo scientific language, that is supposed to disguise the subjective bias of the reviewer.
I am always amazed that there are individuals who seem to be impressed by the length of the review (erroneously described as “depth’).
Geoffrey Crawley tried to review a lens with at most a hundred words and he thought that would be enough to give a serious profile of the reviewed lens. He also refused to add illustrations because they are easy to misinterpret or to put too much value on some phenomenon.
The best test for an optical system is still the brick wall and a slide film.

Manufacturers of photographic equipment know that any product today has a limited production time with a general lifecycle: A product announcement is hyped by the ubiquitous cheerleaders, publishing on the social media. Then a product is available and the sales are initially high. Then sales flatten and even dry up. The product managers know this and they carefully hold back some improvements for inclusion in future model changes. When a new market is required, a totally ‘new’ product is designed. ‘New’ is not the correct designation: the competition has these features already or a clever modular construction design allows for a mix of features. Canon is a master in this area.
Leica follows a different strategy. Given the high investment in a new or improved model and the long period of amortization (required because of low production volume), a new model can be announced every two or three years (compared to the six months to twelve months that a typical Japanese manufacturer requires). As soon as a camera model approaches the flat sales cycle, the company starts producing special models, specially for the M-range.
This was the obvious sales strategy for the period between 1980 and 2000 when a confusing amount of special editions was announced.
Not one of these special models could improve or efficiently support the photographer to take better pictures (technically and/or artistically). This statement is true for the most current special M models. While previous editions used iconic models of the past as a source of inspiration, current special editions are fashion statements, like the haute couture world.
The most recent edition is the Kravitz Monochrome Drifter edition, one of the most appalling and ugly versions in the history of the special editions. The ASC edition occupies a good second place, because of the simulated gold plating of the lenses. The subtleties of previous models are gone and the design statement is more aggressive and may I say (?) desperate.

Most Leica aficionados are familiar with the (iconic) picture of Rodchenko (Girl with a Leica) from 1934. Recently a book with the same title has been published in 2017, written by Helena Janeczec. It is about Gerda Taro, the friend of Robert Capa. She was a very good photographer herself, better even then Capa. She died in the Civil War in Spain. The story is simple. Capa got a second-hand Leica from a friend, but hardly used it. He gave the camera to Gerda, who used it all the time. Capa himself used the Contax camera and also had a Rolleiflex with him.
The Leica people stubbornly insist that the Capa pictures (the well-known “dying soldier” is one of them) were made with a Leica, which is not true. It is a fact beyond doubt that the D-day pictures were made with a Contax. The story behind these pictures is rather fascinating and AD Coleman is one of the main investigators to find the truth. It seems that the actual heroism of Capa is less impressive than the myth. And the poor guy in the darkroom did not screw up the rolls of films as is often claimed.
Even the dying soldier may be a staged scene. No one knows for sure.
The book about Taro is a very pleasant read. It seems to be available in Italian (the original) and in a Dutch translation, which I am reading.







By the way:

All orders for my new book, the Leica Path, have been shipped.