While trying to uncover what the data may be saying, one must have a ‘prior’ or some expectation of what one might see from the data. For example, a friend woke up one early morning and turning westwards, found the sun rising! Something was amiss, he concluded; since his (as well as every one else’s) prior was that the sun rises from the east. He reasoned therefore that it must have been the full moon sinking in the west and enquired whether it was a full moon the previous night. Indeed he was correct. He did not proclaim that the sun rises in the west sometimes!
One might argue that this was an easy case since the sun rising in the east is an established fact. But people often reject their priors without due consideration. Some years ago, a very well known economist ran into my office claiming an important discovery. He had been trying out an analysis of Indian data through Computable General Equilibrium Models and claimed that as income went up, consumption went down. The relationship between income and consumption was not the positive one but that it was negative and so that destroyed many things among other things, the Keynesian Multiplier, for example. I did try to point out that there could be something erroneous in his formulation but he stormed out in anger at my closed mind refusing to see what was plain. After a few days he did call me up to say there had been a problem in the formulation and that we could breathe easily and stick to the positive nature of the relationship between income and consumption.
I was reminded of this strange behaviour of analysts who ditch their priors, during the exit polls conducted for the recent elections in Bihar. During the presentation of one of the most elaborate exit polls, the anchor repeatedly kept on saying how things were turning up against their expectations, formed during a very extensive tour of Bihar. They continued finding tortuous explanations of why their expectations were not holding up. I was immediately reminded of the income-consumption relationship and wondered whether this was just a case of some one making a complete ass of oneself. It turned out that this was precisely what happened.
What should they have done? It is easy to say in hindsight of course, but I will present an argument I wrote to a rather distraught colleague in Chennai the night the poll result was announced before the election results were announced. I did say that the exit poll results would be unlikely to be replicated in the actual voting. For, I argued, the exit poll to correctly depict the voting results, three quite unexpected things would have to happen:
- Mr Manjhi’s joining the BJP would have to have a substantial effect.
- Women voters would have to vote for the BJP
- Events outside the State and comments made by various people in the Central Government which passed off without any public rebuke from the leadership would have no effect.
During the discussion it was repeatedly pointed out that these would have to happen and that surprised the analysts since each one went against their priors. As I told my colleague in Chennai, any one of them could happen of course but all three? Little chance. As it turned out, none of these happened and the exit poll was totally erroneous.
Statistical Inference is difficult of course. The Bayesian method which is adopted by many consists of beginning with a ‘prior’ and then on the basis of the observation, one revises the prior to obtain which is called, naturally, a ‘posterior’. If the revision is drastic, then there must be one of the two: either the prior was formed carelessly or there is some problem in the conducting of the exercise. Or is it that they felt that there was no drastic changes in their prior? In the light of what actually happened, either way they goofed and did so badly.
Again, as the context will make clear, this was written several years ago, immediately after the Bihar state polls in 2015 when the Mahagathbandhan won. Still worth a read I thought and hence retained.
Anjan Mukherji 