Part 2: How We Think Differently
The Problem
I experienced firsthand the frustration (see Part 1) of wildly-inaccurate predictions made using cutting-edge market research methods (adaptive and choice-based conjoint analysis, which were SOTA in early 80s). When I was a newly-minted BCG partner in the heady days of the 80s, we capitalized on the arrival of Apple and IBM PCs to utilize these highly-rational methods to determine optimal design and to predict demand for new products. The PC made surveying tolerable and the matrix algebra feasible – the results were clear and statistically reliable. What could possibly go wrong?
Our predictions were what went wrong – seriously wrong! We over- and under-estimated demand, often by enough to invalidate our earlier recommendations. I checked and re-checked the analyses and the research methods – impeccable. Highly frustrating, especially for BCG which was then the citadel of rational analysis backing competitive business strategy.
The Solution (“I Was Afraid of That”)
After maybe a third failed product launch which research had told us would succeed, I began to question our assumptions. Were we missing critical causal data and so making lousy predictions about how consumers and business people would behave? I looked at my own experience with the “non-rational” side of thinking and concluded that we were ignoring tons of important and influential stuff. People use their instincts, feelings and intuition in decision-making, on top of and sometimes instead of rational costs and benefits. There was growing awareness that these non-rational cognitions operated differently and independently from rational thinking. (See the rise of Behavioral Economics beginning with Kahneman and Tversky.) And awareness that rational and non-rational thinking interact in complex ways in the decision-making process. (See Part 1 of “How We Think Differently”)
If this is true – if non-rational thinking plays a pivotal role in decision-making – we need to understand lots more than just the rational preferences and choices, if we’re going to predict behaviors accurately. But how to research the “soft side” of thinking?