Data: Helpful, Useless or Misleading?TO MAKE MANAGERIAL (BUT ALSO POLITICAL) DECISIONS, IS IT BETTER TO RELY ON INSTINCT OR ON THE SCIENTIFIC METHOD? THE PANDEMIC HAS SHOWN THAT THE ANSWER IS IN HOW WE USE THE DATA
by Alfonso Gambardella, Head of the Department of Management and Technology, Full Professor of of Management, Bocconi University
We have learned many lessons from COVID-19, and many more we are going to learn. I want to focus on one: management under uncertainty. This squares with the soaring availability of data and tecnologies for managing and processing data. The pandemic has shown that data can be helpful, useless, or misleading, depending on how we make decisions and use data.
I am part of a research group that advocates that management should become more “scientific” in the sense that managerial decisions under uncertainty should follow the same approach that scientists use in their research. Many managerial decisions involve no uncertainty. They are decisions about how to achieve a known goal more efficiently – e.g. how to minimize costs to produce a given quantity of output. But decisions that involve growth, such as innovation, or responses to competitive threats, are decisions under uncertainty. While management has taught us a lot about the first type of decisions, the same is not true of the second type. We should also beware of extreme views: artificial intelligence and computer science will solve most managerial problems, or at the other extreme, gut feelings is the only way to go.
A scientific approach to managerial decisions advocates that managers have to first develop a theory of what they expect. They then have to use data to test their theories and take actions based on the outcomes of these tests. As an illustration, I played a game in one of my classes. I told two students that they were going to earn 100 points per supporter of Inter FC in the class if they made an investment that costs 800 points upfront. The question is whether to make this investment. They developed a theory based on the number of male students, students from Milan, and other potential predictors. They predicted, respectively, 18 and 6 Inter supporters in the class. I then asked them to take a sample of 15 students out of the 100 in the class, and they found that 2 (or 13.3%) supported Inter FC. Both the test and the students’ original gut feelings matter because they updated their prediction to 14 and 10. On the one hand, their predictions converge; on the other hand, gut feelings may embody information that we do not account for in the tests. Who is right or wrong? It depends on who had the best gut feelings, or in the language of statisticians on “priors”. However, the important point is that, compared to the initial gut feelings, the test improved both decisions: there were 12 Inter supporters in the class!
If you think that this is not real, here is another example. A company has used “big data” to predict that customers who pay their bill online are more reliable (pay earlier and they are more likely to pay). But what to do with this prediction? Only a theory can inform about actions. Customers who pay online could be more reliable because of their characteristics or because online payment is easier. This yields two different managerial actions: monitor offline payers or diffuse online payments. The company could study two identical populations that for random reasons paid online or offline. If they observe differences in reliability, the company ought to extend online payment, otherwise it needs other experiments to understand what affects payment reliability. A similar example is the prediction made by Scott Stern and colleagues at MIT that used data on all the start-ups in Massachusetts to show that start-ups with shorter names are more likely to grow. This is useful for venture capitalists who need to predict high-growth companies, but not for entrepreneurs because if they shorten their names they are unlikely to grow faster. In order to understand how to grow faster, they need theories and tests that inform them whether it is a good idea to take the actions suggested by their theories.
Managers still rely too much on gut feelings and use data to make descriptions (“here is how sales are going this year”) or, more rarely, predictions. The big opportunity is to use data for making decisions using theories and by desiging tests that identify the value of the actions they theorize. Gut feelings help the scientific management of decisions. For example, if our prior is that online makes payments easier and do not find good evidence of it, we run another test to be sure; if we have the opposite prior, we save the cost of the new test.
This also puts Universities and business education in the right perspective. Too often we hear of genius entrepreneurs who made it only thanks to their intuitions: but we never hear of the many more that did not make it for the same reason. Universities and business education are, and will remain, the locus in which we learn the method for making good managerial decisions, whether in firms, government or other organizations. The COVID experience has shown that this is truer if we use and manage data efficiently.