The ExplanationScience.org community has started

Fellow skeptic and avid blogger Tracy Allison Altman from Evidence Soup and Ugly Research recently started a new initiative called Explanation Science. She asked me if I would like to join here initiative, which I naturally did. But, why did she start this initiative and what is it about? In this short article she answers a few of my questions. To be included on the distribution list of Explanation Science, send your name, location, and work email address to this address.They’ll keep you posted on new developments




So Tracy, why did you start this community?
I started ExplanationScience.org after years of observing (and obsessing over) how people develop, communicate, and apply evidence when they want to make change happen. I realized that although evidence matters a great deal, how we explain that evidence is equally important. So I decided to pioneer a new field: Explanation Science. Explaining is one of the most important things people do, but it hasn't received specific management attention or R&D focus. 'Explaining' can happen anywhere, and can be done by anyone: When we are problem-solving, debating/arguing, teaching, discussing results with investors or stakeholders, announcing research findings, selling products, etc.

This new membership organization is dedicated to the 'science of explaining'. It's a place for people who want to know what works (and what doesn't): To improve the process of explaining, whether it's finding innovative ways to develop valid explanatory information, or finding better ways to present a persuasive explanation.

For several years, I've been an outspoken advocate for processes and technologies that enable evidence-based management. I see www.ExplanationScience.org as complementary to initiatives supporting "evidence-based _____". If we can help people gather better explanatory information, and help them provide better explanations, then we can help them drive wider adoption of evidence-based management methodologies.

What is the aims and/or goals of the community?
We want to advance the idea that explaining is a specific type of activity that's crucial to success - and that it can be purposefully examined and improved upon. We want to build an active online community, and also sponsor face-to-face events. We're planning a kick-off event in Denver, Colorado during Q1 of 2010, and a conference in northern California later next year.

We want to give people a place to put their heads together: To discuss methodologies for gathering explanatory evidence, to exchange case studies on presentation strategies, to evaluate technologies, and to publish findings.

Our vision statement says "Explaining is one of the most important things people do, so we're giving it the attention it deserves."

Who participates in the community?
We welcome people from all walks of life who want to do a better job of explaining, and better understand what belongs in a ‘good’ explanation. People who are asking questions like:

- What information do we need to explain customer buying behavior?
- What techniques are best for explaining a particular health outcome or environmental impact?
- Which technologies are best for presenting explanations, or searching for them?

Our individual members are people interested in the 'science of explaining': Such as technologists, analytics experts, business professionals, scientists, educators, policymakers, analysts, or marketers. We'll also have institutional/corporate members, to serve two purposes: To help fund our group's activities, and to introduce the 'science of explaining' to people inside those member organizations.

Why is it interesting for potential clients?
We offer an opportunity to contribute to the development of a new field: The 'science of explaining'. People with fresh ideas can make a real difference by joining our group. We're providing a place to demonstrate expertise and share experiences on an important topic. We'll provide numerous ways to participate: Leading (or contributing to) online conversations and organized debates. Publishing applied research. Speaking at conferences. Contributing to a knowledge base. Showcasing technologies.

We're bringing together people who otherwise wouldn't be connected: People from different focus areas, industries, and geographies who share a common interest. Our online community, publications, and technology showcase will appear at www.ExplanationScience.org

Thank you Tracy, talk to you later.

Use your illusion

Progress by the current school of thought of evidence based/informed decision making is slow, but steady. That's the good news. One thing I've learned so far, is that evidence does not speak for itself. Contrary to evidence based medicine, random control tests (the golden standard in research) in management situations are difficult if not impossible. Achieving the highest level of evidence in general has to be ruled out. Evidence has to be appraised, which makes it subjective. It is all about the context, interpretation and the quality of the data. Bob Sutton was also musing on this in his posting on intuition vs. data driven decision making: some rough ideas. Our brains are primed on previous experiences. (That's why we don't need all the syllables in a sentence before we interpret what is being conveyed). We humans are pattern seeking animals. This has all kinds of evolutionary benefits (hence intuition, gut feeling, etc.), but also some major drawbacks. To put it mildly, we suck at judging facts. That's the bad news. This raises all kinds of philosophical issues on knowledge, epistemology and ontology, but I don't want to discuss that right now. To illustrate how bad we are at appraising information, you can read books like Fooled by randomess and Black Swan by Nassim Nicolas Taleb or the Halo Effect by Phil Rosenzweig. But, it is easier to watch this clip of a speech by Beau Lotto called Optical illusions show which he recently gave at TED.

Jeffrey Pfeffer on the state of affairs of evidence based management

Last week I received mail from Stanford. Jeffrey Pfeffer had returned after some extensive traveling and found the time to answer my four key questions. Here you can read his take on the current state of affairs of evidence based management. I'm always grateful if people take time out of in their busy schedules to help me out.








1. What do you view as the core idea and purpose of EBM? What do you think would be the benefit for organizations and society as a whole if management would be based more on evidence?

The core idea is simple and seemingly common sense to make decisions, to the extent possible, on the basis of the best facts and evidence available at the time, and to update ideas and decisions as new evidence becomes available. The purpose is to make sounder decisions by basing choices on the best theory and facts instead of on belief or ideology, unreliable retrospective consideration of experience, and fads and fashion. The benefits could be profound, as at least in the United States, in domains ranging from criminology to education to business management, many unsound decisions are made wasting resources and affording poor outcomes.

2. What progress has the EBM movement currently been making and is there evidence of evidence based management getting a foothold in organizations?

I think progress has been halting and even in medicine, there is evidence of much backsliding. For instance, when data showed that radiological screening for breast cancer should be postponed until age 50, there was an outcry and there seems to be no will to implement these findings, even though they are based on a lot of science. In management, senior managers seem to trust consultants, their own judgement, and casual benchmarking much more than they do the facts.

3. What future do you envision for EBM in (research and practice)?

In research, we need in management the same sorts of efforts that have occurred in other fields--summarizing the state of knowledge (and updating such summaries) so they can be used by practitioners. We also need more research on the barriers to implementing EBM practices and how to overcome such barriers. In practice, we need collectives and associations that push EBM forward, much as there have been in other fields.

Thank you very much for your time and efforts.

Miguel Olivas Lujan take on evidence based management

Just last week I received a nice email from Miguel Olivas Lujan. He just became aware of this little blog and was kind enough to attach his article on evidence based management, which was not yet in my collection. Miguel is professor of administrative sciences at Clarion University in Pennsylvania. The article is titled Evidence Based Management: A Business Necessity for Hispanices which he published in The Business Journal for Hispanic research. Subscribers to this journal can read the whole article here. Non subscribers can send an email to Miguel for a personal copy. (this prevents any copyrights infringement on my part...). Since he wrote an article on evidence based management, I thought it would only be fair to ask him some fundamental questions on evidence based management.

1. What do you view as the core idea and purpose of EBMgt?
The core idea in my opinion is –first of all—the systematic use of research methods in managerial decision making. But the decision maker’s judgment and the integration of stakeholders’ values are also essential components that have often been overlooked. Managerial problems are often so complex that decision makers have to extrapolate (also known as making “best guesses”) from previous experiences because a solution has not been tried in a particular context. Also, there are many areas (take for example, international management) whereby stakeholders’ values (particularly customers’ and employees’ values) are such that “what has been found” in one context might not work in another (a classic example would be managing by objectives or MBO). The core purpose is making managers and organizations more effective and rational, less wasteful, yet respectful of differences and mindful of personal competencies and weaknesses (yes, I’ll admit this sounds much as a “pie in the sky” but I’m an idealist at heart).

2. What do you think would be the benefit(s) for organizations and society as a whole if management would be based more on evidence?
Greater efficiency, less politics (well, of the “spinning wheels in vain” kind; I agree that the use of evidence per se will have political repercussions), accelerated progress, more competitiveness…

3. What progress has the EBMgt movement currently been making?
At this point, I believe we have been able to start a much needed conversation. I wish I could report a greater impact but (well, speaking from the academic experience) it’s not easy to dedicate the time this movement deserves when teaching our students is what brings bread to the table. Now, the conversation is not “without teeth” either! Denise Rousseau, along with David Denyer and Josh Manning have raised the standard for literary reviews; Sara Rynes has been documenting specific problems and making actionable recommendations within the Human Resource Management profession; Joan Pearce just wrote a new, and already award-winning textbook focusing on research findings; Jean Bartunek, Gary Latham, Sara Rynes, and others have individually been writing on how academicians can better communicate their findings to practitioners, etc. Plus, the fact that you, Tracy Altman, Bob Sutton and Jeff Pfeffer have been bringing this topic to the “blogosphere” (I probably should include my blog here too) also attests to the fact that there is an important need to be satisfied. But, as Denise stated in her response to this question, it may take “a generation before a new evidence-informed practice takes hold.”

4. What future do you envision for EBMgt in (research and practice)?
More focused work! The need is too important to be left unattended! As my article in the Business Journal of Hispanic Research suggests, if other fields such as Medicine and Education have been able to make such significant inroads through the Cochrane and the Campbell collaborations, why not Management? It is evident that the ride will not be smooth as the inertia to do things the old way is quite strong… but, as we say in Spanish, “Roma no se hizo en un día” (Rome was not built in a day) ;-D

Thank you very much Miguel for sharing your insights.

Richard Feynman's skeptical view on social science

The never ending debate on "real science" and "pseudo science". In this short clip, Richard Feynman points out the 'fallacies' of social science research. Interesting to watch.

The Halo effect

This morning I received the book The Halo Effect by Phil Rosenzweig. It is recommended by Nassim Nicolas Taleb. On the cover it reads "One of the most important management books of all time". Well, you can't blame book publishers for lack of trying to get a book sold. The under title of the book is intriguing. It is called ...and the eight Other Business Delusions That Deceive Managers. Of course, we don't like to admit how little we know. The social psychologist Eliot Aronson observed that people are not rational beings so much as rationalizing beings. We want explanations. We want the world around us to make sense. (p. 12).

Managers don't usually care to wade through discussions about data validity and methodology and statistical model and probabilities. We prefer explanations that are definitive and offer clear implications for action. We like stories.( p. 15). Maybe that's why management is (still) not an academic discipline like geology or medicine. There is nothing wrong with stories, but some stories (like the constant flood of business books) are dresses up as science. They take the form of science and claim the authority of science, but lack the rigor and logic of science. They're better described as pseudo science.(p. 16). The famous Cargo Cult Science of Richard Feynman is used as illustration. Evidence based decision making is for the manager who does not go for the quick fix. Many manager complain they have so little time to take informed decisions. But there is of course a trade off. Good decisions taken at a slower pace or bad decisions at a rapid pace. This point is also made by professor Rob Briner in his video using evidence based management in your day job.

The halo effect
The halo effect is a way for the mind to create and maintain a coherent and consistent picture, to reduce cognitive dissonance ( p. 50-51). Why is it so hard to understand why some companies succeed and others fail? In fact, our thinking about business is shaped by a number of delusions, the first of which is the Halo effect. So many things we-managers, journalists, professors and consultants- commonly think contribute to company performance are often attributions based on performance. And even if we try to gather data in large-scale examples, like the Fortune survey or the Great Place to work study, we often do little more than multiply the Halo effect. (p. 64). Great examples are given with cases studies on Cisco and ABB. When the S&P ratings are up, journalists, professors and consultants tumble over each other to applaud and offer praise for the vision and steady leadership of the CEO. However, when a year later the stock market collapses, the same leaders are blamed for ruthlessness, ego mania and lack of customer focus. Other delusions include the mix up of correlation and causality, the delusion of single explanation (anecdotal evidence), connecting the winning dots (selective sampling, i.e. only picking winners), lasting success (cookbook recipes), absolute performance (blueprints for success?), wrong end of the stick and finally organizational physics

The famous studies In Search of Excellence (Peters and Waterman), Good to Great (Collins and Porras) and Build to last (Collins and Porras)are critically appraised.It is off course easy to point out flaws, but what lessons can be learned from these mistakes? For one thing, there is no holy grail for high performance organizations (yet). Finding evidence to support management decision making is not at all easy. These examples just illustrates the gap between management and science. Furthermore, putting evidence in context, avoiding delusions and understanding the evidence (if available) is a team effort. Even then, it is still difficult and it is no guarantee for success.

What do I think of this book?
The book is easy to read and the examples (Lego, Cisco, ABB, Microsoft) are well chosen. I agree with the message of the author that managers should be skeptical, if not critical of management fads, business books and self proclaimed guru's who supposedly know the magic formula for lasting success or a high performing organization. On the other hand, it is pretty demanding to expect from a manager to look for "the evidence" if it is hard to find let alone being able to appraise it. As a researcher I have my work cut out for me.

Grinded nails and bikes in your cereal

Last week, the Chair of the UK Government’s Advisory Council on the Misuse of Drugs – Professor David Nutt – was forced to resign. The Council was set up some time ago as an independent body to advise the UK Government on its drugs policy by providing evidence about the harmful effects of drug use. In particular, a drug classification system has been in use which is intended to reflect the level of harm of different drugs. Some years ago the UK Government ignored the advice of this panel and moved the classification of cannabis into a higher harm category to “send a signal” to users. Of course people at the time asked if there was evidence that doing this did deter users or not. I don’t think anyone knows and in fact some argue it might make it more attractive.

David Nutt had in the past criticised that decision and this week questioned the fact that alcohol and tobacco were not included in this classification system and suggested they were worse in terms of their harm to individuals and society than ecstasy, LSD or cannabis. He also suggested a new classification system. This seemed to be too much for the Government and they saw it has his attempt to change policy rather than just offer evidence-based advice and sacked him.

Here in the Netherlands we have a consumer program called 'De Keuringsdienst van Waarde', the title is a paraphrase on what in the UK is called the Food Standards Agency. The literal translation would be 'The value agency'. Four research journalist investigate every episode a specific type of food. E.g. orange juice, peas, onions, tea, etc. Two weeks ago, they had a topic on Kellogs Special K, with added iron. Ironically, the voice over said that the stuff of grinded nails and bikes is added to the cereal. They asked a food professor from Wageningen University, a family doctor and a chemist at Corus (the iron ore processing factory). None of them believed actual iron was added to the cereal. It must like iron in spinach, like Popeye.....or so they thought.

The journalists conducted tests (with metal detectors and magnets) and it turned out that the cereal indeed has magnetic properties. Then the fun really started. They added water to the cereal and put it in a blender. Next they put a magnet in a plastic bag and surprise, surprise, they found small iron particles. To the amazement of everybody, including the food professor, the family doctor and the chemist at Corus. Strangely, Kellogs did not want to answer questions on camera. The production process of cereal is proprietary information.....

Home movie to extract iron from your cereal

Time for serious research journalism. Why is metallic iron added to cereal? Is this good for public health and more important is this legal? All our Dutch food agencies (i.e. the government) point toward each other and then point to the EFSA (European Food and Safety Authority). (Note. Nobody checks for evidence). The EFSA doesn't know anything about approving nor stating that iron should be added to food, let alone metallic iron. In the final shot they are talking to the Dutch Food Authority who say, well if this is true, the product will be banned!

That was two weeks ago. A media storm broke out. Kellogs bought advertising space the next day, claiming that the program misrepresented the whole process of adding iron to food and actually blamed the journalists for bad journalism and slander.The voice over tells us that they were completely wrong with their metaphor two weeks ago about grinded nails and bikes, the should have said grinded cars... Kellogs has an iron supplier in Denmark who also supplies iron ore and powder to the car industry (a script writer could not come up with this...). All of course according to the highest standards....The food professor was on the receiving end of critique by his peers and was buried with reports and research. Only to find out that all the reports contradict each other! The poor man was intimidated and didn't want to participate in the program anymore. But, he had a collegue who would. This man clarified that adding iron to food stems from the USA, where it is mandatory by law since the second world war. The body can only absorp 9% (max.) of this type of added iron in an acid (the stomach) environment. But Kellogs claims it is needed because 40-60% of the men, women and children receive too little iron in their daily diet. It's good for you. They do it in the interest of the public and the claim their way is approved and our Dutch Food agency takes the side of Kellogs.... (so much for consumer trust in the authorities...).

It turns out that predecessors of the ESFA scientists (20 years ago) were highly doubtful of the effects of iron as food supplement. The never approved it. So, the health claim of Kellogs is false. And, in Denmark the practice of adding iron to food is forbidden by law, because people get sick and die as a result of to much iron in their food (hemochromatosis, type 2). Kellogs has a similar product (Special K) in Denmark, without the added iron. The cereal boxes look very much alike. We learn that Danish people are the only ones who think for themselves.... (sadly). The lady from the Danish consumers union states that supplements are abnormal and points out the obesity epidemic in Europe (and the USA). This episode ends with a chat at the dining table at the Dutch consumers union. They are appalled about the findings and are planning on launching a campaign against food supplements. The food professors announced new research. What is the scientific evidence of benefits from metallic iron as food supplement? One can only wonder what our politician will do. At least they can't fire the journalists.

To make a long story short, managers are certainly not the only ones in the dark about evidence based decision making. The poster boys for circumstantial decision making turn out to be the agencies who are there to protect the general population from health hazards.