Post

Understanding Artificial Neural Networks: Mysterianism about Known Mechanism is Mysticism Olivia Guest, Nancy Abigail Nufiez Hernandez, and Mark Blokpoel Mysterianism is the idea that human cognition, mind, cannot be understood. Taking this concept and applying it to known mechanism — such that claims are made that we do not know how engineered systems, such as artificial neural networks (ANNs), work, or that they constitute black boxes that we can only open with difficulty — is inappropriate at best and malicious at worst. We do know the mechanistic structure of such models because we designed and built them. We also do know their functional role (what they are for) as well as the mathematical function they are asked to approximate (map inputs to target outputs). Because mysterianist beliefs about known systems, such as ANNs, are often expressed, scientists need to sit up and take notice. We provide an error theory as to what is going on to help unpick this metatheoretical blunder. Ultimately, the problem is that ‘understanding’ is not a technical term in these cases: the word is co-opted for a specific narrative to sell ‘artificial intelligence’ through mystification. All computational systems, from pendulums to databases, will behave in ways we cannot predict or control — this is not a unique property of ANNs — and experts do indeed grasp the computational properties of these systems nonetheless.

Figure 1 Three co-occurring (latent) assertions in the context of con- temporary reasoning on Al and cognition. While they may appear to hang together in “reflective equilibrium” (Elgin 2017), their metatheoretical examination brings their contradictions to light. Starting from the top, and moving clockwise: a) represents the typical assertion of so-called cognitive, neural, or otherwise biological plausibility or inspiration presented for ANNs; B) is the claim we tackle head on here — both in terms of analysing the constituent terms and in how it relates to the other two assumptions; and y) exemplifies a type of mysterianism, which while not always explicitly expressed, plays a background role and importantly comes into conflict with the other two claims in important ways.

Figure 3 Not all systems, S; relate to their relevant data, D; and world phenomena this way, but ANN models (as many inferential statistical models) do when used without scientific understand- ing of the data-generating process or without verbal or formal theory to match (Guest and Martin 2025b, 2021, 2023). As we move from left to right above, we move from full understanding of engineered systems, S; to less or even possibly none of the data, D; to the world which can be thought of as the set of phenomena to be understood.