The wide view of the Beijing Olympics as seen in the wealth of newswire stories about the event, reveals a near flawless product. If there were a gold medal for macroergonomics, the organizers could make a convincing claim to it. But the wide view isn’t entirely truthful. At closer range, ergonomics and human factors issues are discernible, and several marred the event.
With efficiency corporations could emulate to their benefit, the Beijing Olympic Organization (BOC) directed hundreds of thousands of workers through separate but complementary multi-year goals. These included infrastructure improvements, the construction of the venues, the beautification of public areas and designing the opening and closing ceremonies. Between these two events that can only be described as triumphs, the BOC ran 302 competitions while meeting the needs of 10,500 athletes from 205 countries, 3 million visitors and billions of television viewers.
For political reasons, it is unlikely the Chinese people or foreigners will ever hear about
behind-the-scenes issues during the preparations. An enterprise on this scale must have had its share of construction fatalities and injuries.
But once the event started, the world was in Beijing. It became impossible for the Chinese authorities to hide some types of problems – as the many headlines about the split lip of taekwondo referee Chakir Chelbat attest. The few headlines that were not about athletes’ wins and losses were usually about faulty judging. In fairness to the BOC, international federations that govern individual sports brought the judging weaknesses to China. The BOC inherited them.
Swedish taekwondo referee Chelbat needed stitches to repair his lip after being attacked by Cuban taekwondo champion Angel Valodia Matos over a judging dispute. Matos and his coach have been banned from the Olympic Games for life.
Judging disputes erupt in sports such as javelin throwing, running and swimming, where performances are measured by photographic or electronic “eyes,” but they are much more common in competitions that can only be judged subjectively. The best examples are taekwondo, gymnasitics, diving and boxing at summer Olympics and figure skating at winter Olympics. There are often extreme disagreements over the results. One judge sees a dive as an 8.0, for example, while most of the other judges score 9.0s and 9.5s.
The Chelbat incident capped a series of controversial taekwondo judging decisions, including the initial failure to award Sarah Stevenson the victory the Briton was said to be due after her +67kg quarter-final against Chen Zhong of China.
After the incident, Yang Jin Suk, secretary general of the World Taekwondo Federation, announced that an official inquiry would be held into the judging of the Stevenson versus Chen bout. There were, he said, “three human factors in judging that we cannot cure: intentional mistakes of referees and judges, accidental mistakes and mistakes from ignorance”.
Bias, particularly grounded in nationalistic issues and politics, is another common human factor that is blamed in faulty judging. Corrupt judges also find their way onto Olympics judging panels, but Beijing appears to have been spared the usual corruption scandals.
If the rules of any sport in the subjective realm and the system for resolving disputes between the judges are poorly designed, then all these human factors-related vulnerabilities are compounded.
A second panel of six officials from the international federation’s technical committee oversees finals in gymnastics competitions and some other subjective sports. If the system of rules or dispute process are poorly designed, there is little the panel can do but multiply the problems inherent in the disputed judgment.
Then there are the disputes, confusion and uneven rules over duplicate medals. Two medals of the same color are allowed in several different sports but not others. Two silvers were awarded in track and field in the women’s 100m dash, for instance, but American gymnast Nastia Liukin had to settle for silver in one gold medal tie because the rules are different in gymnastics. She was part of the silver-winning American team and won bronze in the floor exercise as well as the coveted all-around gold. But on the uneven bars competition, she was awarded silver despite tying on 16.725 points with China’s He Kexin, who was awarded gold.
As there was no difference in the routines’ difficulty value, the gold was given to He on the basis that she had 0.033 fewer deductions for execution, despite having taken a small step on her dismount.
In 2004 the International Olympic Committee (IOC) rejected South Korea’s request to have a duplicate gold medals awarded in men’s all-around gymnastics because of a scoring mistake that gave the title to American Paul Hamm. Judges incorrectly scored Yang Tae-young’s parallel bars routine, failing to give the South Korean enough points for the level of difficulty. Yang ended up with the bronze but would have won the competition if the scoring had been correct.The international gymnastics federation, known as FIG, apologized for the mistake and suspended three judges, but said it couldn’t change the results under its rules.
In the same year the IOC also turned down an official German petition for duplicate gold medals following a judging error in three-day equestrian competition: dressage is another sport that is judged subjectively.
"There was a huge Korean push to have a second medal, and a huge German push to have a second medal, but we said no," said IOC president Jacques Rogge. "You can’t stop national Olympic committees from asking for more medals; this will never stop. The IOC will be consistent and say no."
But the IOC is not consistent. At the 2002 Winter Games in Salt Lake City, the IOC awarded duplicate gold medals to Canadians Jamie Sale and David Pelletier after a French judge said she was pressured to put a Russian couple ahead of them in the pairs skating event. Rogge reiterated that the judging cases in Athens are not comparable to the Salt Lake scandal. "The IOC is very consistent in its position: as long as the federation gives a result, and as long as there is no proof of manipulation or corruption, we will accept the result," he said. "We accept that human error is unavoidable in judging and refereeing."
After the 2004 furor, Rogge said the IOC will meet with the federations after the games to consider ways of improving the judging. He noted that boxing and figure skating have adopted computerized scoring systems to eliminate subjective judging, and suggested other sports could do the same. "FIG already agreed they will have to change their way of judging," Rogge said. "We’ll discuss with them how."
If FIG did improve the rules after the Athens Olympics, the improvements didn’t prevent the many disputes in gymnastics at the Beijing Olympics.
Few experts agree with Chang’s and Rogge’s assessment that the biggest factors amount to incurable problems.
Many continue to ask why it would be such a problem to award duplicate medals or to continue the competition, where it is absolutely necessary to break the tie, by requiring a tiebreaking routine of three or four required elements.
Rule changes made by the IOC and national federations since the spate of judging scandals at the Winter Olympics of 2002 seem designed to eliminate the nationalism that often biases judging in subjective competitions. The new rules do not allow judges from any of the countries who have participants in the finals to judge the finals.
One commentator sees flaws in this change, arguing that “we end up with incompetent judges from places I have never heard of or places that do not have a strong gymnastics program.” Why not have a judge from every country in the finals, he said. That way the possible "advantage" would be balanced out, or neutralized.”
In so many words, one expert argues that neutralizing bias could be a mistake.
The factors in judging subjective sports underlie issues that human factors and business experts resolve routinely in corporate workplaces. In a 2003 research paper Eric Zitzewitz, Assistant Professor of Economics at the Stanford University Graduate School of Business in California, looked at the 2002 judging scandals for lessons that can be applied to corporate situations. He wrote that forcing decision makers to boil their opinions to a single vote can encourage bias and vote trading. Controversially, he doesn’t scorn either. Turning opinions into votes actually can make vote trading easier, he argues.
“When designing a group decision-making process for potentially biased managers, intuition tells us that throwing out extreme opinions neutralizes the most severe biases among the group’s members and leads to better decisions.” He notes that new research suggests “our intuition may be wrong.”
He pointed out that figure skating judges are nationalistically biased. As a response to these biases, he said, figure skating under-weights extreme opinions more than any other judged Olympic sport. “It sounds like a good idea to throw out extreme scores when judges may be biased, but this actually makes things worse. Throwing out extreme scores throws away a lot of information.”
Using techniques he developed to study stock analyst forecasts, he found that extreme opinions have information, noise, and bias in the same proportions as less extreme opinions. “Those extreme opinions are not as biased as we think. The same is true for analysts. When an analyst says something very different from the pack, listen!“
The next Olympics is set for Vancouver. Figure skating and other subjective competitions will again be in the headlines. If the IOC and national federations have learned anything from the judging disputes at Beijing, the 2010 Winter Olympics could be the first where judging disputes are too insignificant to make the headlines.
Sources: Associated Press; Reuters; Beijing Olympic Committee; Athens Olympic Committee; Salt Lake Olympic Committee; Stanford University
This article originally appeared in The Ergonomics Report™ on 2008-09-03.