Don't confuse de facto with de jure

Interesting example on the importance of context in deciding on measurement. Also a great example of the importance of using direct empirical evidence over abstract proxies. From Is the World Bank’s ‘Doing Business’ Report at Odds With How Business Is Done in the Developing World? by Raymond Zhong.

The World Bank’s ”Doing Business” report, an influential survey of the operating environment for companies in nearly 200 countries, doesn’t accurately reflect the experience of companies actually doing business in developing nations, a new study has found.

The annual publication from the development lender assesses the ease of compliance in 10 facets of private enterprise, from starting a business to getting electricity to paying taxes. The results are used to generate a much-publicized country ranking, which governments around the world take as a yearly checkup on their economic stewardship.

Singapore flaunts its top spot, while laggards turn improving their rank into national priorities. Retaking the Russian presidency in 2012, Vladimir Putin vowed to vault Russia to 20th by 2018 from 120th in 2012. The country came in at No. 62 this year. India’s prime minister, Narendra Modi, testifies to his business-friendliness by promising to take his nation, 142nd in this year’s report, into the top 50 by 2017.

But Mary Hallward-Driemeier, a World Bank economist, and Lant Pritchett, a Harvard University economist, find “almost zero correlation” between the Doing Business findings and those based on surveys of business enterprises that the World Bank helps conduct around the world. On average, the amount of time companies tell surveyors they spend on three tasks—obtaining construction permits, getting operating licenses and importing goods—is “much, much less” than the times recorded in the Doing Business report.

The divergence, the authors hypothesize, stems from the gulf in poor countries between the laws and policies that exist on the books and the ones that prevail—or perhaps don’t prevail—in reality.

The Doing Business report “does not summarize even modestly well the experience of firms as reported by the Enterprise Surveys,” Ms. Hallward-Driemeier and Mr. Pritchett write in the latest edition of the Journal of Economic Perspectives.

Mr. Pritchett said in an interview that for developing-country policy makers, focusing on rising in the Doing Business ranks could draw scarce resources away from more-substantive reforms that would help the government better administer and enforce business regulations.

“The pretense that Doing Business measures the real rules, and that if we just modestly improve these Doing Business indicators, they would somehow become the reality of what the rules are and how business is really done—I think that’s a very dangerous fiction,” he said.