U.S. Open random draw questioned
ESPN analysis finds top two seeds had easier first rounds than statistically probable
An "Outside the Lines" analysis of 10 years of men's and women's Grand Slam draws shows the top two men's and women's seeds in the U.S. Open -- on average -- faced easier opponents in the first round than is statistically probable if the draws were truly random.
Not only do both of the men's and women's first-round U.S. Open matchups deviate significantly from true randomness, this skewed pattern was not found at the Australian Open and Wimbledon, which use a similar draw system. At the French Open, the difficulty of opponents for the top two women's players during that time period was significantly more difficult than a random draw should produce, but the men were in line.
USTA Pro Circuit Director Brian Earley, who has been the U.S. Open tournament referee since 1992 and presides over the draw, said he stands by his system. However, he said he was concerned about the questions the analysis raises about the random nature of the draw.
"I have such faith in the folks within my work that if there was something unfair about it, I think it probably would have been proven to me and to the tournament before this," he said. "But we are always interested in hearing input."
"Outside the Lines" analyzed the average difficulty -- determined by the players' ATP or WTA rankings before the draws -- of those who played the top two seeds in all Grand Slams over 10 years. That was compared to 1,000 random simulations of 10 years of Grand Slam draws -- or the equivalent of producing 10,000 random draws taken 10 years at a time.
Only three of OTL's 1,000 simulations produced first-round opponents as easy as those the top two men's seeds have actually faced on average over 10 years in the U.S. Open. In none of the 1,000 simulations did OTL get the extreme results found in 10 years of actual opening matchups for the top two women's Open seeds.
Dr. Andrew Swift, past chairman of the American Statistical Association's Section on Statistics in Sports and an assistant mathematics professor at the University of Nebraska at Omaha, said the analysis and its methodology were sound.
"Any way you want to look at these, there is significant evidence here that these did not come from a random draw," he said.
That finding didn't surprise Scoville Jenkins, who in 2004 was ranked 1,433rd in the ATP singles rankings when he scored a wild-card entry into the U.S. Open. That made him the lowest-ranked player among the 128 entries in the men's tennis tournament. His opponent in the draw? No. 2 seed and defending champion Andy Roddick.
"At the time you think, 'Wow, this is unlucky,'" he said. "There's so many players in the draw I could have played."
A truly random draw for the unseeded players -- which is promised by USTA officials -- should have given Jenkins a two-thirds chance of playing another unseeded player, and a roughly 31 percent chance of playing a seeded opponent outside the top two seeds. He had a 2.08 percent chance of facing a top-two seed.
After facing Roddick in the first round in 2004, Jenkins drew No. 1 seed Roger Federer in the first round in 2007, when Jenkins was the 125th-best player in the tournament, according to the OTL analysis. He lost both times.
"Sometimes I think they put the player against who they would like to play," said Jenkins, who has since retired from professional tennis and is now an assistant tennis coach at Kennesaw State University near Atlanta. "If somebody came out tomorrow and said, 'This whole time we weren't doing it random and we were picking by whatever system,' it would not surprise me."
After being presented with the "Outside the Lines" analysis, Swift conducted his own study of the opponents of the top two seeds and found that only four times in 1 million simulations did he come up with an average ranking equal to or easier than what was actually observed in the men's and women's draws over the last 10 years.
"By itself, the U.S. [Open] numbers are weird," he said. "And then they're also weird in comparison to the other three Grand Slams. So you've got a double argument of weirdness here. Something weird is going on."
[...]
"What would the U.S. Open gain by fixing the draw in this way? I believe the U.S. Open would gain nothing," Earley said. "I think that that would be a risk that the U.S. Open would never take. Never."
