You may have heard of The Hawthorne Effect, a phrase that lives on as one result of a series of studies that took place between 1924 and 1927 at the Hawthorne Works of Western Electric Company in Cicero, Illinois. This article is a review of "Shining New Light on the Hawthorne Illumination Experiments," an article appearing in Human Factors (2011), by Masumi R. Izawa, Michael D. French, and Alan Hedge, through which they describe "an historical and statistical analysis of archival data from the Hawthorne illumination experiments." The researchers relied on copies of the "original documentation and much of the data from all three illumination experiments," which had previously been thought destroyed, but which had actually found its way into an archive at Cornell University.
Note: HFES and its publishing partner, SAGE, is offering free access to all HFES periodicals, including the article reviewed herein, through the end of November, 2012. Details appear at the end of this article.
Drawing on previously published accounts of the controversial Hawthorne studies, Masumi et al provide a fascinating 'who dunnit' review of their background:
Masumi et al conclude their historical review with this statement, arguing that none of the studies they reviewed conduct a comprehensive analysis of the results of the three original experiments:
More recently, Levitt and List (2011) claim to have uncovered the original data from the illumination experiments and conducted a statistical analysis to verify the elusive Hawthorne effect. A close review of their study reveals that critical details of the illumination experiments were glossed over.
This statement left me wondering what Levitt and List claim to have uncovered, and what they "glossed over," but Masumi et al do not provide any additional detail.
Masumi et al also claim to have access to original documentation, authored by Snow, that ultimately found its way to Cornell University. They report the path to Cornell as:
The Original Experimental Methods
Reviewing the archived materials, Masumi et al found that Snow had reached his conclusions by "eyeball analysis of vector graphs that plotted the percentage changes in production against lighting treatments and light levels." In the 1920's, statistical theory and methodologies where not very sophisticated, so Masumi et al first sought to understand the experimental designs, then applied appropriate modern statistical tests to the available data. The authors provide a great deal of detail in the full article, and interested readers are encoured to read it.
Next, Masumi et al extracted and organized original data from the graphs and tables contained in Snow's draft reports and re-analyzed the data using modern statistical methods, which they describe in detail in their paper. I will not review the detailed findings here, but for those with a solid interest in and understanding of experimental design and statistical analysis, it is a worthwhile read. Instead, I will now turn to Masumi et al's findings and interpretations.
The First Illumination Experiment
Contrary to Snow's original conclusion that light level did not affect productivity, Masumi et al found that in one experimental treatment there was a positive effect but that positive effect did not repeat in the second, similar treatment. The authors attribute this to a potential seasonal effect, because the experimental treatments took place at different times of the year, each with different lengths and intensity of natural daylight that could have had an influence. "Unfortunately, it is not possible to further disentangle these effects because of the limited data collected and summarized by Snow," they report.
The Second Illumination Experiment
Snow reported that there was a positive effect on productivity, but believed 'that the effect of increased supervision and the psychological factors incident to test conditions are of such magnitude as to completely mask any effect of illumination on production.' On the contrary, Masumi et al's statistical analysis shows no significant effect on productivity, and other than Snow's suggestion, they were unable to recover any data surrounding the question of supervisory changes.
The Third Illumination Experiment
Snow had concluded that, 'This test in a attempt to find the critical point of illumination . . . has demonstrated that the operatives [workers] will maintain their productive output at that level or in the neighborhood of that level which is customary, even in the face of insufficient illumination and the attending discomfort.' In other words. Masumi et al's analysis, however, found "that test group productivity was higher as the difference between test and control group light levels was greater (i.e., test group light levels were lower)," a statement that is unclear to me. At another point they say, "A negative association between light level and productivity was found in the third experiment."
So, in summary, Masumi et al's analysis found that there was:
They suggest that these inconsistent results could be due to confounding factors, including uncontrolled work-related issues, experimental design issues, data factors, and the lighting levels that were tested in the experiments.
Lighting Level Issues
The lighting levels used in the different experiments varied, and the levels in experiments two and three were "mostly" lower than in experiment one. The highest lighting levels in experiment three, for example, were 1/5th the level of those in experiment one, and in all experiments, the lighting levels were lower than modern standards would dictate.
Experimental Design Issues
In order to adequately compare productivity across the three experiments, the durations of the lighting treatments in each would need to be consistent. However, the treatment periods ranged from 1 day to 24 days. The 1 day duration was driven by management concern that the lighting level tested would affect employee relations; the 24 day treatment was driven by the complexity of installing and uninstalling the lighting used for that particular treatment.
The layout of the building could also have affected the experiments, because each was conducted in different locations featuring different daylight exposures from windows. Compounding this effect, Snow notes that changes were made to the window treatments during the experiment, modifying the effect of daylight mid-experiment. The first two experiments, due to their building locations, exposed workers to a mix of daylight and artificial illumination. In the first experiment, lighting fixtures were also modified mid-experiment to reduce glare. Taken separately or together, these types of mid-experiment modifications confound baseline and experimental condition comparisons.
There were also inconsistent lighting levels at different periods in experiment one due to variations in power voltage. Masumi et al also noticed that task lighting was available to some workers in the Inspection department, though it is unknown if they were used during the experiments.
Psychosocial changes were also introduced mid-experiment in experiment three with the abrupt introduction of increased supervision.
Snow's log notes indicate lighting levels were sampled at different times of the day and at different locations. However, the individual measurements data was not contained in the archive, so Masumi et al had to rely on Snow's summary data consisting of daily group averages and general lighting levels, substantially reducing the power of their statistical analysis.
Masumi et al point to several important issues that could have significant effects on these experiments, including:
Even though the different types of tasks likely required different lighting, each was treated the same in the experiment.
So, What is The Hawthrone Effect?
Masumi et al note that The Hawthorne Effect has been interpreted in different ways at different times since the original experiments were conducted in the late 1920's. For example:
Masumi et al suggest that, based on Snow's own writings, it was originally tied to the simple realization that the experiments, by their very nature and in the way they were carried out, increased supervision. Their research also suggests that the phrase Hawthorne Effect first appeared in writing in 1950, when John R. P. French Jr. wrote:
The experiments did demonstrate in a dramatic way, however, the importance of social factors in group productivity. From a methodological point of view the most interesting of these findings was what we might call the ‘Hawthorne effect’.
Regardless of where the phrase originated, they caution against interpreting it to mean that lighting does not affect productivity, due to the many study limitations described in their analysis. In their conclusions, Masumi et al state:
Our analyses of the Hawthorne illumination experiments have shown the value of understanding the historical, organizational, and business context in which this work was undertaken …
I enjoyed reading and reviewing this article, because the topic and story is full of intrigue and has direct impacts and implications on ergonomics. It's amazing to me that after all these years, we are still debating the studies themselves, as well as their meaning and application. It's truly a cautionary tale of what can happen when preconceived expectations influence scientific discovery; how important experimental design, data collection techniques, and interpretation methodologies are to our understanding of the world around us; and what happens when evidence is destroyed in an ill-conceived attempt to cover-up the truth, whatever truth that may be.
What specific truths the Hawthorn studies may have provided has been a subject of debate ever since, and speculation, interpretations and anecdotal accounts can get us only so far. Masumi et al's review of the history and analysis of the treasure trove in the Cornell archives takes us further toward the truth, yet still leaves much mystery.
It still leaves us wondering … does lighting affect productivity? If so, in what ways, and how could we measure and prove it in an industrial setting?
Masumi R. Izawa, Michael D. French, Alan Hedge, Shining New Light on the Hawthorne Illumination Experiments, Human Factors, October 2011, vol. 53, no. 5, pp. 528-547, doi: 10.1177/0018720811417968
At the time of this writing, this article was accessible online to HFES members and SAGE subscribers at: http://hfs.sagepub.com/content/53/5/528.full
HFES Nonmembers: Free access to HFES periodicals for the month of November 2012! Gain access at: https://online.sagepub.com/cgi/register?registration=HFES112012
This article originally appeared in The Ergonomics Report™ on 2012-11-07.