How much has psychology left to discover? When I ran a poll on this question a few years ago, most respondents were optimistic about how much knowledge future research could yield:
But I’m more pessimistic, at least when it comes to the parts of psychology that describe us as agents – personality, reasoning, how we interact with each other – with little reference to the brain.1
Why psychology is easy
One reason is that we already know a lot about human psychology. Most academic research concerns things that are hard or impossible to observe directly: the small (particle physics, genetics), the distant (astronomy), and the past (history, palaeontology). By contrast, psychologists study individual human behaviour, which is relatively straightforward to observe. As a result, many important patterns have already been identified, whether long ago through everyday observation or more recently through formal research.
Why psychology is hard
But clearly, psychologists still don’t understand human behaviour perfectly. So why haven’t we made more progress? In my view, a key obstacle is that important constructs such as conscientiousness, deliberation, and risk aversion refer to loose clusters whose boundaries are anything but sharp. This means that we cannot establish psychological regularities with a high degree of precision, and that many go undetected altogether. And I don’t think that better research practices could make this issue go away. The problem is the bewildering complexity of the underlying phenomena. Our constructs cannot capture all of this complexity, and should be seen as rough approximations.
Beyond psychology
Though the limits to progress are especially pronounced in psychology, they appear throughout the social sciences. There is certainly a lot of useful applied and fine-grained work to be done, but the big-picture questions have been so extensively studied that I expect further advances on them to be relatively modest (at least until the advent of superintelligent AI).
Optimism about the social sciences’ potential is often paired with a negative view of their current state. “If only researchers weren’t so sloppy, we could make much more progress.” But I think this betrays a poor sense of proportion. While methodological reforms are welcome, they’ll only make so much of a difference. Social scientists aren’t as irrational as the negative narrative suggests. The main reason we don’t make more progress is simply that the low-hanging fruit has been picked.
From here on, I’ll use “psychology” to refer to these parts.
Minor comment, concerning "By contrast, psychologists study individual human behaviour, which is relatively straightforward to observe." Not sure that phrasing is entirely adequate. Wouldn't many psychologists argue that they (also) study what humans think. Often probably with regards to effects on behavior, but not necessarily so. One might even argue that observing what goes on in the human mind is more difficult than observing genes or particles, or even distant stars - in fact, we have measured the latter, while only having a very crude approach to measuring what happens in the mind (e.g. we can correlate certain neurons to certain images). Maybe you would just argue that what I have in mind is cognitive science, and thus distinct?
The construct problem is one Kahneman circles without naming.
Most findings in social psychology are correlations between two imprecise operationalisations, and what gets reported as a stable effect is often a stable relationship between approximations.
That's part of why replication is hard.
You can't reliably reproduce something that was never measuring a determinate thing.
The low-hanging fruit wasn't only the easy experiments, it was the ones where the construct had a thick enough core that the slippage didn't matter.