Can a machine reason?

Section 1The Problem of Reasoning.

One of the interesting things people have found about AI models, particularly LLMs, is that they have some very strange biases and quirks. For example, LLMs have trouble spelling things, rather ironic given they’re large LANGUAGE models, but it actually makes total sense once you realize how their architecture works.

They also tend to struggle with common riddles that have been slightly altered; they tend to default to the common answer, even though it’s wrong.

They also struggle with paradigm shifting within context. Once they lock into an idea, they tend to keep at it for the rest of their context.

These quirks have led to people implying that LLMs don’t actually reason, and they are finding correlations and patterns. Real reasoning, people have said, is causal reasoning. Humans can reason about cause and effect, but machines cannot.

However, if you think about it, Causal reasoning is not a different type of thinking from correlational thinking. It’s still just correlation, merely used with more sophistication, on the correct data.

Humans are not actually good at causal reasoning either. The typical person reasons correlationally most of the time; this is why post hoc arguments are so common. You often need rigorous education and training in reasoning to go beyond this simplistic form of reasoning.

Section 2Pearl’s Ladder

Judea Pearl in 2018 described three rungs of causal reasoning:

1. Association(seeing): Noticing that people who take a drug recover more often. This is correlation.
2. Intervention(doing): Let’s intentionally give some people some drugs and see what happens
3. Counterfactuals(imagining): “This person took the drug and recovered, but what if they hadn’t? What would have happened?”

Pearl argued that most machine learning stayed on the first rung; they were just finding patterns in data and could not climb to rungs 2 and 3. Many AI critics are still using some variation of this argument, but it’s from 2018, forever ago in AI research.

Meanwhile, Rungs 2 and 3 are still correlation, just done with care and data selection.

For example, intervention? You carry out certain actions and observe the correlations. You give the drug to some people and not others, and you record what happens, then you look for a correlation between “given the drug” and “recovered”

What about Counterfactual reasoning? It just involves having a long set of correlational data in your mind and reasoning about them. You can reason about what would correlate with not using a drug by knowing what correlates with not using similar drugs. If you have no data at all about this type of situation, you cannot deduce an outcome in your counterfactual.

And, of course, this large correlational data is a model. It’s compression; it’s understanding. AI systems that build models are, in principle, able to answer counterfactual questions about them.

Section 3Humans Also Falter

Humans are amazing, but. Humans do not have godlike abstract causal reasoning capacity; it’s mostly learned, and when not careful, we commit to quick reasoning that’s confident but can be wrong.

There’s a lot of evidence for this,

For example, Helena Matute and others ran an experiment where people see a button and a light. The light comes on at random, and the button has no effect. People still thought the button controlled the light, especially when the light came on often.

Consider various forms of superstitions. Ritual sacrifices for wealth, the belief that a cold comes from cold weather, or that the cranky old woman who lives down the street has superpowers and is an evil villain.

Or consider Gazzaniga’s Interpreter: In split-brain patients, the two halves of the brain cannot communicate. Michael Gazzaniga showed a snow scene to the right hemisphere only, and the left hand chose a shovel. Then he asked the patient why. The left hemisphere did not see the snow, but it made up an explanation anyway. Gazzaniga called this system “the interpreter,” which produces a causal story after the fact, with full confidence, whether the story is true or not.

Humans have pretty impressive causal perception; it helps us navigate the world, but note that these are heuristics, not abstract reasoning.

Section 4Good Causal Reasoning is Recent.

Reliable causal knowledge is actually the result of thousands of years of improvement in reasoning techniques and experimentation. Consider, for example, that Doctors used to bleed people for over 2000 years. They would just cut you and let you bleed out. They saw some patients get better after it and concluded that it worked. This is entirely rung 1 and perhaps rung 2 causal reasoning, and humans held on to it for centuries.

Hume, in the 18th century, argued that we never see causes directly. We only see that one event regularly follows another; then the mind joins them together causally. The idea of cause is inferred, and when things are not actually causally linked, it can lead to incredible amounts of delusion. Consider, for example, the numerous instances of human sacrifice that humans have done over the years, for absolutely nothing.

Controlled trials started emerging slowly in the second millennium BC, but the first modern randomized controlled trial emerged less than 200 years ago.

Humans have had brains like ours for 300,000 years, but for almost all of that time, until less than 100 years ago, we ran on poor causal reasoning for a lot of abstract concepts beyond our everyday lives.

Section 5Causal Reasoning is Refined Correlational Reasoning.

There are formal methods that can find causes from patterns of correlation. If an algorithm can do it, then it cannot be special to human minds.

For example, consider Reichenbach’s common cause principle: if A and B are correlated, then either A causes B, B causes A, or a third thing, C, causes both.

For example, Ice cream sales and drownings are correlated. But both are caused by hot weather.

In 1993, Spirtes, Glymour and Scheines turned this concept into algorithms. The PC algorithm looks at which variables become independent when you condition them on others.

1. Measure correlations
2. Measure conditional correlations(correlations in groups)
3. Use the pattern to find causal structure.

You can also run into collider bias and direction, which again require more information or looking at correlations differently in order to resolve. There is nothing here that isn’t resolved with more data or different assumptions. A machine fundamentally can access all of them.

It turns out that causal reasoning by itself can be modelled, and thus, can be understood, and because learners are understanders, any general learner can learn to do causal reasoning.