Beware the Hawthorne
“We have great news for you. Our project is delivering results already.” The team is all smiles when they give this update during the project meeting. The carefully prepared graphs unveil a remarkably shorter time for the whole process, from customer request to delivery of results. “We have applied a hypothesis test and the result is significant with a p-value of flat zero!” They sound like they know what they are doing.
When asked for the change in the process, they all give different answers. When asked what the root cause to have achieved this effect was, their smiles fade. “We actually only implemented some Quick Hits. They turned out to have a greater effect than we thought. Isn’t this a nice surprise?” They asked.
If the going is real easy, beware, you may be headed downhill. Anonymous
When addressing process issues, there are different constellations to describe the relationship between improvement solutions developed and the outcome:
Effective Solutions that show Excellent Results. This is the dream of every project leader. This does not only refer to success. It also leads to the assumption that the solution has addressed the root causes for the problem and in turn, generated the expected result.
Seemingly Effective Solutions that don’t show Expected Results. There is a multitude of potential reasons for this. It always means going back to the drawing board and searching for the gap, be it initial wrong assumptions, biased data, insufficient analysis or solutions implemented the wrong way.
Toothless Solutions that don’t show Results. This certainly does not come as a surprise.
Toothless Solutions that deliver Excellent Results. This constellation does sound unlikely. However, it is not. Sometimes, even without implementing any solution, the process improves and results get better – often to a remarkable degree.
This can be attributed to the Hawthorne Effect. Hawthorne says that observing a situation will change the same. It is bound to appear in all environments where humans are “watched”.
In our experience, it is even worse. Talking about a process is enough to trigger Hawthorne results because people running the process get more aware, focus more on their work and deliver better results. In some situations, only then they do what they are supposed to do. The Hawthorne Effect and results go away as soon as people are no longer “watched” and the attention is directed towards other things. In this sense, Hawthorne results are not sustainable.
Hence, the apparent disadvantage here is that real project results that are driven by the changes in the process cannot easily be distinguished from the ones caused by the Hawthorne effect. It becomes an extra variable in a situation where only one variable should exist.
The only way of making sure that improvement effects will last, is to establish a logically consistent chain from description of the problem via potential root causes, data collection and analysis to identification of the vital few tackled by related solutions. Lean Six Sigma is what these steps are. Follow them carefully and you will be successful – beyond Hawthorne.