Something within me takes control of my right hand and writes down the solution to the problem I have been thinking about. I don’t understand the solution as I’m writing it down, and only later, after having thought about it for some time, do I understand it in some nebulous way. Further thinking and checking the solution points to it being correct.
This kind of possession occurred to me when I was trying to solve math problems at school, including during my final high school exams. Having come up with a correct answer without fully understanding it almost felt like cheating. It also happened during my PhD, when I wrote code. In one instance I remember, I needed to implement a series of recursive function calls to compute and draw phylogenetic trees, but how exactly they fit together to make the whole script work was beyond me. I tried for several hours to come up with an answer, until the solution wrote itself, again without understanding coming first.
The only way I can describe what reached out through my hand is intuition, but that’s not a satisfying explanation of what’s going on. I wonder if comparing our minds to large language models is instructive. Our experiences are equivalent to the LLM’s training data and our memories, at least the implicit ones, are equivalent to the LLM’s parameters. Our learning occurs continuously, while LLM training occurs in massive, one-off training runs. In this comparison, the chain-of-thought the LLM provides about its thinking process is equivalent to our consciousness, maybe in the way described by Nick Chater.
Russ Roberts, on his EconTalk podcast, recently interviewed mathematician David Bessis. Bessis has written a book about the importance of intuition in mathematics. He believes that intuition is highly trainable. What happens when we learn mathematics is that we train our intuitions. I’m sure this also happens in other fields, but mathematics (and programming) has the advantage that it’s possible to immediately check our intuition to see if it was correct, making for faster feedback and therefore faster learning.
Nehaveigur 