“We have great news for you. Our project is delivering results already.” The team is all smiles when they give this update during the project meeting. The carefully prepared graphs unveil a remarkably shorter time for the whole process, from customer request to delivery of results. “We have applied a hypothesis test and the result is significant with a p-value of flat zero!”
They sound like they know what they are doing. When asked for the change in the process, they all give different answers. When asked what the root cause to have achieved this effect was, their smiles fade. “We actually only implemented some Quick Hits. They turned out to have a greater effect than we thought. Isn’t this a nice surprise?” They asked. Or, is it just a Hawthorne?
If the going is real easy, beware, you may be headed downhill. Anonymous
Ensure a Cause and Effect Relationship
When addressing process issues, there are different constellations to describe the relationship between improvement solutions developed and the outcome:
Different Solutions and Corresponding Results
Effective Solutions that show Excellent Results. This is the dream of every project leader. This does not only refer to success. It also leads to the assumption that the solutions have addressed the root causes for the problem and in turn, generated the expected result.
Seemingly Effective Solutions that don’t show Expected Results. There is a multitude of potential reasons for this. It always means going back to the drawing board and searching for the gap, be it initial wrong assumptions, biased data, insufficient analysis or solutions implemented the wrong way.
Toothless Solutions that don’t show Results. This certainly does not come as a surprise.
Toothless Solutions that deliver Excellent Results. This constellation does sound unlikely. However, it is not. Sometimes, even without implementing any solution, the process improves and results get better – often to a remarkable degree.
When Toothless Solutions Deliver Results
This can be attributed to the Hawthorne Effect. Hawthorne says that observing a situation will change the same. It will likely appear in all environments where humans are “watched”.
In our experience, it is even worse. Talking about a process is enough to trigger Hawthorne results because people running the process get more aware, focus more on their work and deliver better results. In some situations, only then they do what the SOP says. The Hawthorne Effect and results go away as soon as people don’t feel “watched” any longer and the attention is directed towards other things. In this sense, Hawthorne results are not sustainable.
Hence, the apparent disadvantage here is that we cannot distinguish between real project results due to changes in the process and those due to the Hawthorne effect. It becomes an extra variable in a situation where only one variable should exist.
Go Beyond Hawthorne
The only way of making sure that improvement effects will last, is to establish a logically consistent chain from description of the problem via potential root causes, data collection and analysis to identification of the vital few tackled by related solutions. Lean Six Sigma is what these steps are. Follow them carefully and you will be successful – beyond Hawthorne.
