A general framework for understanding how the causal structure of some systems cannot be fully captured by even the most detailed microscale description

An interesting proposition. From When the Map Is Better Than the Territory by Erik P. Hoel. From the Abstract:

The causal structure of any system can be analyzed at a multitude of spatial and temporal scales. It has long been thought that while higher scale (macro) descriptions may be useful to observers, they are at best a compressed description and at worse leave out critical information and causal relationships. However, recent research applying information theory to causal analysis has shown that the causal structure of some systems can actually come into focus and be more informative at a macroscale. That is, a macroscale description of a system (a map) can be more informative than a fully detailed microscale description of the system (the territory). This has been called “causal emergence.” While causal emergence may at first seem counterintuitive, this paper grounds the phenomenon in a classic concept from information theory: Shannon’s discovery of the channel capacity. I argue that systems have a particular causal capacity, and that different descriptions of those systems take advantage of that capacity to various degrees. For some systems, only macroscale descriptions use the full causal capacity. These macroscales can either be coarse-grains, or may leave variables and states out of the model (exogenous, or “black boxed”) in various ways, which can improve the efficacy and informativeness via the same mathematical principles of how error-correcting codes take advantage of an information channel’s capacity. The causal capacity of a system can approach the channel capacity as more and different kinds of macroscales are considered. Ultimately, this provides a general framework for understanding how the causal structure of some systems cannot be fully captured by even the most detailed microscale description.

When you model something in order to forecast, you are inevitably removing detail in order to make the model function better. You are converting the territory into a map.

It seems inherent, with less detail, that the map must be less accurate than the territory even though its utility can be greater. Perhaps, but . . .

Hoel is approaching this from a neurological context whereas my experience is primarily in operational and sociological forecasting.

I have long recognized that there seems a contradiction between the loss of information inherent in modeling and the fact that modeling is useful.

My explanation has been three-folld.

The map removes noise and allows better visibility of the essential nature of the terrain which might not otherwise be obvious.

In a deterministic macro system (X always leads to Y), deterministic subsystems can establish patterns of aggregate macro-system patterns that are independent of the nature of the subsystems and therefore the macro model is forecasting macro patterns not aggregations of subsystems.

In a complex/chaotic macro system, multiple deterministic subsystems may have different error rates which cancel each other out, yielding a non-deterministic but reliable macro-system pattern.

But what happens when you have a complex/dynamic/chaotic macro system whose constituent components are also complex/dynamic/chaotic? Those are the hardest to model because all patterns are emergent and often short-duration non-repetitive. In that context, how does a simpler model work?

Hoel introduces additional ideas:

5. Causal Emergence as a Special Case of Noisy-Channel Coding
Previously, the few notions of emergence that have directly compared macro to micro have implicitly or explicitly assumed that macroscales can at best be compressions of the microscale [19,20,21]. This is understandable, given that the signature of any macro causal model is its reduced state-space. However, compression is either lossy or at best lossless. Focusing on compression ensures that the macro can at most be a compressed equivalent of the micro. In contrast, in the theory of causal emergence the dominant concept is Shannon’s discovery of the capacity of a communication channel, and the ability of codes to take advantage of that to achieve reliable communication.

[snip]

In comparison, the theory of causal emergence can rigorously prove that macroscales are error-correcting codes, and that many systems have a causal capacity that exceeds their microscale representations. Note that the theory of causal emergence does not contradict other theories of emergence, such as proposals that truly novel laws or properties may come into being at higher scales in systems [31]. It does, however, indicate that for some systems only modeling at a macroscale uses the full causal capacity of the system; it is this way that higher scales can have causal influence above and beyond the microscale. Notably, emergence in general has proven difficult to conceptualize because it appears to be getting something for nothing. Yet the same thing was originally said when Claude Shannon debuted the noisy-channel coding theorem: that being able to send reliable amounts of information over noisy channels was like getting something for nothing [32]. Thus, the view developed herein of emergence as a form of coding would explain why, at least conceptually, emergence really is like getting something for nothing.

[snip]

The application of the theory of causal emergence to neuroscience may help solve longstanding problems in neuroscience involving scale, such as the debate over whether brain circuitry functions at the scale of neural ensembles or individual neurons [34,35]. It has also been proposed that the brain integrates information at a higher level [36] and it was proven that integrated information can indeed peak at a macroscale [16]. An experimental way to resolve these debates is to systematically measure EI in ever-larger groups of neurons, eventually arriving at or approximating the cortical scale with EImax [3,37]. If there are such privileged scales in a system then intervention and experimentation should focus on those scales.

Hoel is operating far beyond the extent of my statistical and mathematical chops but his ideas are intriguing.

The observation that there is a structured "general framework for understanding how the causal structure of some systems cannot be fully captured by even the most detailed microscale description" is an intriguing one for highly complex and dynamic systems such as macro-societal phenomenon (spread of destructive epistemic belief systems), macro-economics (Hayek's problem of knowledge), medico-biological (why some treatments are effective for some people under some circumstances but not for other people and not under all circumstances), and such other complex dynamic systems such as climate.