Incomplete knowledge, conflicting objectives among the different parties and binding constraints on who has the authority to make decisions

From What do consultants get paid for? by Luis Garicano.  The subheading is The analysis is the easy part.

A consultant I had lunch with recently is redesigning the loyalty program of a large airline. His team finished the analysis in two weeks. Months later the program still does not exist. This is because the purpose of the assignment is not to solve an analytical case study, but to figure out which redesign the parties will accept, and to get the people with authority to commit to implementing it.

I was not surprised to hear the story. In our just-published book Messy Jobs: The Work That AI Cannot Reach, Jin Li, Yanhui Wu, and I argue that a job is not a collection of independent tasks but a bundle of tasks and a position inside an organization. While many of the constituent tasks are clean, the job is messy because they must be combined under incomplete knowledge, conflicting objectives among the different parties and binding constraints on who has the authority to make decisions.

Hence we argue that automating the clean parts does not necessarily eliminate the job, because the remaining activities, tightly bundled with the rest, can remain the constraint. We argue that the bundle is strongest where separating the analytical/cognitive parts that can be automated would destroy local knowledge, trust, accountability or continuity.

This is related to Scott Adams' concept of a talent stack.  

Another point.

The real knowledge problem

An advocate of highly capable AI systems (“AGI-pilled”) would probably say this is a problem ready for AI. Have an agent redesign the program, have it meet the other constituencies, have it come back with a solution.

But what happens in those meetings deserves a closer look. There are four frictions in the room that make the meetings necessary.

First, knowledge is dispersed. This is the Hayek problem: different people hold different bits of information. AI can help with this. By making the emails, contracts, past redemption data and meeting transcripts searchable and available to the system, the model can extract part of this dispersed knowledge. But a lot of the dispersed knowledge remains in people’s heads as it is local and contingent, and will only emerge in the meeting.

Second, knowledge is also tacit: the Polanyi problem. We know more than we can tell. A senior partner in the consulting firm who knows the client well knows instinctively that a particular proposal will not fly. Again, AI may help reduce this problem, as it can learn from the actions people take on the basis of their knowledge. Brynjolfsson, Li and Raymond (2025) found that AI assistance diffused some of the communication and problem-solving practices of stronger customer-support agents to less experienced workers. The system had captured enough observable patterns in stronger agents’ behavior to reproduce some of their tacit knowledge advantage.

The third friction is that knowledge is not available to the AI system because people refuse to disclose it: strategic private information. The hotel knows how much it would cost to eliminate one feature of the loyalty program, and it will exaggerate that cost to extract more value in the exchange. Under specific assumptions about bilateral trade, Myerson and Satterthwaite showed that no mechanism can guarantee full efficiency while also inducing truthful revelation, respecting voluntary participation and balancing the budget. The theorem applies equally to humans and machines. Better models reduce the cost of drafting and bargaining, but don’t solve the problem of deciding who gets what.

The fourth friction is that the objective has not yet been formed or authorized. Think about the hotel chain. If you ask them initially what they want out of their program, they may not have an answer, since the organization does not know which features are critical or the cost of conceding a feature. That exploration only happens through iterative meetings, and what is happening in those meetings is that people are collectively discovering what the organization wants, what the different features are worth and which relationships they care about. The chief executive eventually makes the call, but the parts of the organization arrive at that point with different views and without a settled objective. The participants use those negotiations to discover the trade-offs, form their own view about their preferences among these trade-offs and authorize someone to bind the organization.