Saturday, October 22, 2011

Selection problem in reasoning

Employees who have chosen to join a labour union seem to make less money than their coworkers who have abstained from union membership. So why would anyone join a labour union to begin with?

J.J. Heckman
The above example illustrates sample selection bias, a typical fault in scientific reasoning that was first explicated in detail by D.B. Rubin (I am not 100% sure) and later addressed by J.J. Heckman (worthy of Nobel prize in 2000). The above reasoning is faulty because people who are likely to benefit from union membership join a union, while those unlikely to benefit decide not to join. It happens to be that in general those who earn more are less likely to benefit from the union membership.

Selection bias in academic research
The selection bias (a specific case of the broader problem of 'endogeneity') is discussed in detail in every good PhD program. Much of the advanced statistics (stuff covered after basic and time-series regression models) relates to the problem of endogeneity. Yet, selection bias remains an endemic weakness in strategic management, and probably plenty of social sciences studies.Some academics have joked that if the reviewer does not like an article and would like it to be rejected, (s)he can always complain about selection bias and endogeneity.

I recently read an article in Academy of Management Journal that I really liked, finding in effect that the actively involvement of managers and their tendency to draw in external stakeholders to discuss problems increased both the quality of resulting agreements and the resulting actions.It is a strong and enticing article. Yet, the author neglects to discuss the possibility that managers are unlikely to engage with problems or call in external stakeholders when they are thorny: managers select which problems to attend to based on their likely ability to resolve the issues. Thus, the seemingly self-evident prescription that the more managers and stakeholders engage with issues the better may be false. Indeed, it is easy to see that when problems cannot ultimately be solved, managerial engagement and the involvement of stakeholders can have high cost for the managers themselves - if not the organization.

Why are academics unable to reason correctly and to attend to the selection bias? I think there are three issues. First, it is very difficult to robustly correct for selection bias. We would have far less research done and published if we insisted on controlling for all potential selection problems. Second, the PhD education, while attending to selection problems, also creates a lot of trust into basic methodologies, both quatitative and qualitative. Researchers will always be overjoyed with any novel findings they make, It is not very enticing for us to go and try to decimate our results. Management being such a shitty practically oriented discipline that it is nearly impossible to publish studies that identify relationships and then prove them to be spurious. Finally, researchers often have a good qualitative understanding of their research subjects. When you know the managers and know how they think, you immediately know that the selection bias is not an issue. If you know it is not an issue, you may not think it is worth doing a lot of extra work to prove it conclusively.

Selection bias of managers?
To my best knowledge (not saying much), nobody has really examined selection bias in managerial reasoning. The phenomenon lies under a broader umbrella of 'superficial learning', the idea that managers learn wrong lessons from their experiences. In reality, we do not know the extent to which managers assume causality from mere correlation.The bias would seem likely: managers supposedly imitate the behaviors of their successful competitors, even though the only reason why less successful companies do not behave in the same way lies in the inability of less competitive companies to benefit from the practices.

The question is pretty significant for two reasons. First, selection bias leads to false causal attributions and thereby wrong decsisions. Second, problems resulting from selection bias can be influenced. By drawing attention to problematic causal attributions, managers can either correct their mistakes or at least approach their causal attributions and knowledge with the required scepticism.

How to study the selection bias in real life? I suppose we would require very intelligently deviced large-scale surveys. The next step would be to design laboratory experiments to investigate potential ways to mitigate biases. While neither form of research really appeals to me, I hope someone would investigate this.  

Selection bias and network centrality
When doing my PhD I had some data from a big telecoms firm to examine interpersonal networks within a big R&D unit. I found, along with the prior research, that engineers who had worked with other central engineers created inventions with greater impact within the firm. However, once I looked at the technological domains these people worked with, there was no longer any causal relationship: engineers were well connected if they worked on technologies that were crucial to the firm and the inventions of these engineers had big impact only because of the technological area they operated in. It turned out that the social ties were selected based on the work task, which also explained the apparent 'productivity'.

The data was not rich enough to "disprove" the importance of centrality in explaining 'innovative productivity', and I lost my interest in the whole domain area over time. Yet, my own observations provide a nagging feeling that many effects reported in research are significantly weaker than expected, but our social sciences are terrible in self-correcting themselves.

Wednesday, October 5, 2011

Reflexivity and irony

Philosophers are a reflexive bunch. Not that they are very shiny (in my experience the contarary), rather they tend to think a lot about their thoughts. In management theory and education, reflexivity comes heavily recommended (see e.g. an article on reflexivity in reseach by my friend Nelson and his co-authors). Reflexivity is commonly associated with wisdom, something most of us would consider desirable.

In this blog post, I raise the question whether reflexivity and irony -- key philosophical virtues -- are also potential problems for managers wo must lead actual organizations.

Reflexivity and irony

To reflect is to ask why, to engage in reasoning where existing beliefs are used to justify or reject an action, a choice, a norm, a belief or an assumption. Philosophy has been and largely still is about reflecting on basic issue - why we consider something to be good, why we accept something to be true, and so forth. An intelligent individual reflects, an ignorant one accept the status quo without considering further reasons.

Richard Rorty, my favourite philosopher, has taken reflexivity to a point where most philosophers become uncomfortable. Namely, he convincingly argues that there can be no reason for a philosophical system that we can outright accept. There is no escaping language, in the sense that nothing but experience outside claims can underwrite our premises. With whatever philosophical standpoint we take, we ought to consider it with irony. This is a sense of irony that Rorty associates foremost with Nietzsche, the detached amusement towards the beliefs one accepts himself or herself. We may accept a certain outlook to life, but we must accept that there is no final unquestionable reason to do so (philosophy is in this sense no different from religion, it relies on belief).

Now, the view has not made Rorty very liked among many philosophers. Such is the burden of irony socially. Ironic approach to management studies is likewise warranted - does what we do make any difference? Are our papers really insightful? Has everything mostly not been said (and forgotten)? Are most approaches not only dogmatic continuations from commonsensical observations mystified by charismatic old men? Whatever the answers, these are worthy considerations. But don't expect them to be crowd pleasers on the cocktail event at a major conference.

Should managers be ironic?

Some authors, including Karl Weick, have called for more reflection and "mindfulness" on the behalf of managers. I am not sure it is always a good thing. Management and leadership benefit from confidence. Social action requires unity and permanence. How can hundreds of employees in an organization coordinate their work efforts if there is no uniform and stable understanding of means and ends, of premises and values? In Blink, Gladwell argues that Gettysburg, one of the most famous battles in military history, was lost because of reflection and indecision. Psychologically, individuals need their life to be predictable. Living with a partner who constantly questions and adjusts key life choices would probably be quite horrible.

A truly strong individual might be ironic privately, yet project utmost confidence externally. The self-control of an actor? The benefit of irony is the lack of fear. One who accomplishes to not take ones own position seriously will not be fooled to respect authority when it is not warranted. But without authority, even the authority of one's own knowledge, what is the basis for continued motivation and effort?

Entrepreneurship is particularly an area where scholars have identified passion and persistence to be advantageous. Irony and reflexivity, taken to an extreme, seem antithetical to passion. Indeed, the stereotypical philosopher is a miserable being mired in the fundamentsl doubt, best exempfilied in Sartre's existentialist novels (and perhaps even better by Camus). The entrepreneur is, in Lampel's words, "an optimistic martyr", an individual who chooses not to reflect on potential problems with the knowledge that the battle is more important than the victory. Because of the complexity of our existing beliefs, reasoning will seldom lead to any closure. A manager knows that analysis leads to paralysis because there are infinite facts and choices to reflect on.

Conclusions? I'll reflect on that...

Ironic reflection is the reasonable conclusion of 20th century philosophy, a conclusion that dethrones philosophy from its position as the meta-science, casting what used to be philosophy into history of philosophy, a humanistic curiosity and a source of inspiration devoid of authority. To reflect is to reason more, to be wiser. But managers might need a sort of meta-wisdom that tells them when not to reflect. We all need that actually, not to become the antiheroes of Camus and Sartre. Maybe someone should do a study on thhe downsides of wisdom and reflexivity in management? I'm not holding my breath to see that published and taught in the MBA programs...