PDA

View Full Version : Hawk-Eye is broken


dh003i
07-11-2007, 07:21 AM
The Hawk-Eye system is broken in a number of ways. The 3.6mm is the average error on the calls, which means that some of the errors could be much greater. The creators ought to release what the 90, 95, and 99% confidence intervals are. If the average error is 3.6mm, the 90% confidence interval could be 10-15mm, and even more for the 99% CI. The way the challenge system works in football is the way it should work here: the burden of proof is on Hawk-Eye to over-turn the umpire. That means Hawk-Eye needs to be within a certain confidence interval on that call; if it isn't, it shouldn't count.

If we're going to have it, I have no problem with the players having challenges; however, making calls shouldn't be their responsibility. The officials should be able to review plays as well, and should not defer to asking the players to "challenge" the call. That misplaces responsibility.

I tend to think the entire thing is just a big gimmicky waste of time and money. The speed at which the simulation is shown is slowed down for the entertainment of the audience. Furthermore, the entire thing is just a bunch of calculations, based off of a number of data-points; I doubt it accurately accounts for the spin of the ball or the wind. It was shown to be inaccurate on clay, when it frequently was shown wrong by the marks.

Federer, Nadal, and Safin are all against it. That means that not a single player who's won a slam since 2005 supports Hawk-Eye. That ought to tell you something.

InvisibleSoul
07-11-2007, 10:46 AM
I would also like to see a more comprehensive analysis of how accurate the system is.

Max G.
07-11-2007, 03:36 PM
Wait, so 3.6 mm is the "average" error now? I've always assumed that that was the 99 or 95 percent confidence interval or something. I don't have any good reason to assume it's that and not the average error or the standard deviation of the measurements though. Do you have any link to some actual description of what the number means?

I agree on your points about the way it should be used though, and second the fact that it would be nice to get access to the data.

tHotGates
07-11-2007, 04:17 PM
We've tried Cyclops then upgraded to Hawkeye but problems remain. ATP executives are thinking out of the box for answers .... purported rumors & whispers of private negotiations persist.

http://konstantin.antselovich.com/wp-content/eye_of_sauron.jpg



Try & argue with that.

:-D

D-man
07-11-2007, 05:03 PM
The Hawk-Eye system is broken in a number of ways.
Alright, you don't like it based on your post for four reasons. Let's take a look.
The 3.6mm is the average error on the calls, which means that some of the errors could be much greater. The creators ought to release what the 90, 95, and 99% confidence intervals are. If the average error is 3.6mm, the 90% confidence interval could be 10-15mm, and even more for the 99% CI. The way the challenge system works in football is the way it should work here: the burden of proof is on Hawk-Eye to over-turn the umpire. That means Hawk-Eye needs to be within a certain confidence interval on that call; if it isn't, it shouldn't count. Furthermore, the entire thing is just a bunch of calculations, based off of a number of data-points; I doubt it accurately accounts for the spin of the ball or the wind. It was shown to be inaccurate on clay, when it frequently was shown wrong by the marks.
So your first reason is:
1. It's inaccurate
My counter-arguments would be:
1. It's more accurate than linesmen and umpires. I think so because I've never seen it make as horrendous a call as I've seen humans make. Umpires get wacky and overule good calls on the far side, linesmen can make big mistakes. I don't think this happens with hawk-eye.
2. It's entirely impartial in all respects. Being a computer, it has no biased whatsoever, and cannot be accused of it.
3. It keeps arguments and anger from players to linespeople and umpires. Try arguing with a computer—you saw Fed do it. It just doesn't last the same.
4. The wind and ball spin, as well as ball squash/rotation/skid/ and roll is accounted for at ground impact.
If we're going to have it, I have no problem with the players having challenges; however, making calls shouldn't be their responsibility. The officials should be able to review plays as well, and should not defer to asking the players to "challenge" the call. That misplaces responsibility.
Your second reason is:
2. It should be used more frequently
Ideally, if it were affordable and faster, it should. However, if a player keeps getting challenges wrong, it just shows that player has no right to complain about calls since their perception of calls is crap. Simple as that.
I tend to think the entire thing is just a big gimmicky waste of time and money. The speed at which the simulation is shown is slowed down for the entertainment of the audience.
Third reason:
3. It used for entertainment
Uh.. all of tennis is for entertainment. wouldn't even exist without it.... so what? Players, upon a perceived bad call, used to feel robbed and go beserk/ and majorly tank teh match from being upset. That's NOT entertaining—now all they can do is get ****ed with a machine.
Federer, Nadal, and Safin are all against it. That means that not a single player who's won a slam since 2005 supports Hawk-Eye. That ought to tell you something.
4. Some players don't like it!
What ought that to tell me? Player's don't want it showed just how bad their perception is, cause then they can't complain and throw tantrums and cuss at the umpire anymore. Or else why would they care? It takes money from their prize fund? I can't think of why a player wouldn't want more accuracy and impartiality, unless some how calls where going in their favor?

BobFL
07-11-2007, 05:13 PM
The Hawk-Eye system is broken in a number of ways. The 3.6mm is the average error on the calls, which means that some of the errors could be much greater. The creators ought to release what the 90, 95, and 99% confidence intervals are. If the average error is 3.6mm, the 90% confidence interval could be 10-15mm, and even more for the 99% CI. The way the challenge system works in football is the way it should work here: the burden of proof is on Hawk-Eye to over-turn the umpire. That means Hawk-Eye needs to be within a certain confidence interval on that call; if it isn't, it shouldn't count.

If we're going to have it, I have no problem with the players having challenges; however, making calls shouldn't be their responsibility. The officials should be able to review plays as well, and should not defer to asking the players to "challenge" the call. That misplaces responsibility.

I tend to think the entire thing is just a big gimmicky waste of time and money. The speed at which the simulation is shown is slowed down for the entertainment of the audience. Furthermore, the entire thing is just a bunch of calculations, based off of a number of data-points; I doubt it accurately accounts for the spin of the ball or the wind. It was shown to be inaccurate on clay, when it frequently was shown wrong by the marks.

Federer, Nadal, and Safin are all against it. That means that not a single player who's won a slam since 2005 supports Hawk-Eye. That ought to tell you something.

There are several mistakes in this post. I will point out two:
1. it is NOT an average error
2. you can conclude nothing without running the regression analysis

Useful url:

http://jtsang.blogspot.com/2006/07/technology-in-tennis-hawk-eye.html

Deuce
07-12-2007, 12:27 AM
My counter-arguments would be:
1. It's more accurate than linesmen and umpires. I think so because I've never seen it make as horrendous a call as I've seen humans make. Umpires get wacky and overule good calls on the far side, linesmen can make big mistakes. I don't think this happens with hawk-eye.
Indeed, umpires and linespeople make mistakes. As in every other officiated sport. Why should tennis be different?
2. It's entirely impartial in all respects. Being a computer, it has no biased whatsoever, and cannot be accused of it.
You don't know this. Human beings control the computerized process of HawkEye - ergo, bias is a possibility.
3. It keeps arguments and anger from players to linespeople and umpires. Try arguing with a computer—you saw Fed do it. It just doesn't last the same.
Disagreements with the officials has been part of tennis forever. It is also part of every other sport that I can think of. To remove this element is to remove a large part of the nature of sport.
4. The wind and ball spin, as well as ball squash/rotation/skid/ and roll is accounted for at ground impact.
In theory, yes. Your sentence sounds like it was taken directly from the marketing promos for HawkEye.

Third reason:
3. It used for entertainment
Uh.. all of tennis is for entertainment. wouldn't even exist without it.... so what? Players, upon a perceived bad call, used to feel robbed and go beserk/ and majorly tank teh match from being upset. That's NOT entertaining—now all they can do is get ****ed with a machine.
Where is your evidence that many players 'tank' matches after getting a bad call or two? Your argument has no foundation whatsoever. In fact, getting bad calls inspires some to elevate their game.
The way I see it, HawkEye exists primarily as a marketing tool. It is there to attract today's video generation to tennis. It seems that more and more people require the element of video in order to keep them 'entertained' and keep their attention focussed on something - so HawkEye is simply a video toy that the tennis administrators are using to attract viewers, sell advertizing space, etc.

4. Some players don't like it!
What ought that to tell me? Player's don't want it showed just how bad their perception is, cause then they can't complain and throw tantrums and cuss at the umpire anymore. Or else why would they care? It takes money from their prize fund? I can't think of why a player wouldn't want more accuracy and impartiality, unless some how calls where going in their favor?
The fact is that all calls balance out. Players will get bad calls in their favour and bad calls that go against them - it all evens out. That being the case, even if HawkEye were 100% accurate (which it is not), it would make no difference at all in the long run, because bad calls even out for every player.
So, essentially, when viewed from this perspective, HawkEye is useless, except as a fancy marketing tool and an entertaining toy for some audience members, the majority of whom don't know much about tennis.


..
.
.....
.

keithchircop
07-12-2007, 12:47 AM
i'd be totally in favour of hawk-eye if it were 100% accurate.

edmondsm
07-12-2007, 01:39 AM
The Hawk-Eye system is broken in a number of ways. The 3.6mm is the average error on the calls, which means that some of the errors could be much greater. The creators ought to release what the 90, 95, and 99% confidence intervals are. If the average error is 3.6mm, the 90% confidence interval could be 10-15mm, and even more for the 99% CI. The way the challenge system works in football is the way it should work here: the burden of proof is on Hawk-Eye to over-turn the umpire. That means Hawk-Eye needs to be within a certain confidence interval on that call; if it isn't, it shouldn't count.

If we're going to have it, I have no problem with the players having challenges; however, making calls shouldn't be their responsibility. The officials should be able to review plays as well, and should not defer to asking the players to "challenge" the call. That misplaces responsibility.

I tend to think the entire thing is just a big gimmicky waste of time and money. The speed at which the simulation is shown is slowed down for the entertainment of the audience. Furthermore, the entire thing is just a bunch of calculations, based off of a number of data-points; I doubt it accurately accounts for the spin of the ball or the wind. It was shown to be inaccurate on clay, when it frequently was shown wrong by the marks.

Federer, Nadal, and Safin are all against it. That means that not a single player who's won a slam since 2005 supports Hawk-Eye. That ought to tell you something.

Amen.

I would actually dump the thing all together. In my opinion tennis players shouldn't be *****ing about calls anyway. It's not a game that comes down to the buzzer. If a player doesn't like that his/her shot was called out. Than he/she should not have hit it so close to the line. If you feel like your opponents shot was wrongly called in, than just remember all the times you got a "fortunate" call.

I hate shot spot.

Wannabe
07-12-2007, 02:57 AM
The hawkeye manufacturers openly admit that 3.6mm is the "average error" of the system on the site below:



Click on "tennis" and look under "accurate and reliable". Let us assume that this refers to the standard deviation and that the errors are spread according to a gaussian distribution. This would indicate that the 95% confidence interval is + 7.2mm and the 99% confidence interval is 10.8mm. This is still remarkably accurate. However, I agree with dh003i that the hawkeye system should have a "don't know" option if the ball is found to be within a certain distance of the edge of the line (I would choose 7.2mm), and that, in these cases, the umpire's original decision should stand.

I don't, for example, think that a ball should be called "in by 1mm" as happened in the final at Wimbledon. This means that there is a 39% chance the ball was actually out. The umpire is only supposed to overrule a line judge if there has been "a clear error." A 60% chance of an error is not clearly an error in anybody's language. Why should hawkeye be allowed to overrule a line judge without the kind of certainty an umpire has to have? I would suggest the reason hawkeye does not have a "don't know" option is that people are afraid that it will not be trusted if it admits to being imperfect. Are audiences really that immature that they are willing to believe something that pretends to know everything rather than something that admits what it does and does not know.

I'm still, in principle, in favour of hawkeye though as it should prevent "bad calls" i.e. unambiguously incorrect ones.

BTW, if anyone concluded from the TV replay that the ball that started the row was clearly out, look at the rebuttal on the site. I found it pretty convincing.

Wannabe
07-12-2007, 02:58 AM
That's weird, my link vanished. Here it is again:

www.hawkeyeinnovations.com

thejackal
07-12-2007, 05:05 AM
even if the margin of error is not that great, at least it evens out, right? i mean pro players do hit shots that graze the line pretty often, so if both players use their challenges then it should at least be "fair" during matches.

AAAA
07-12-2007, 05:41 AM
even if the margin of error is not that great, at least it evens out, right? i mean pro players do hit shots that graze the line pretty often, so if both players use their challenges then it should at least be "fair" during matches.

No. The machine can and does make bad calls just like people.

Fries-N-Gravy
07-12-2007, 06:20 AM
instead of the 6 cameras or whatever and computer estimation/animation. since it already costs so much money, they should've just installed 600 cameras around the stadium and use REAL replay and analyze it from the different angles. think the matrix type of camera "movement". can't argue with real footage

sureshs
07-12-2007, 07:41 AM
instead of the 6 cameras or whatever and computer estimation/animation. since it already costs so much money, they should've just installed 600 cameras around the stadium and use REAL replay and analyze it from the different angles. think the matrix type of camera "movement". can't argue with real footage

There is such a thing called cost.

Eviscerator
07-12-2007, 07:43 AM
I would also like to see a more comprehensive analysis of how accurate the system is.

I agree...

sondraj
07-12-2007, 07:59 AM
O.K so fed complains about one stupid call and now hawke eye is broken. He is a drama queen and so is the tennis world for following in his footsteps

Kim
07-12-2007, 08:13 AM
Oh my God, another troll (sondraj). Federer a drama queen? You sure you know your definitions??

Tanner77
07-12-2007, 08:17 AM
As said from the commentators...

Hawk-eye does make some errors, and has been overuled before because it was way off.

But HUMAN ERRORS ARE THERE TOO! so why not be smart and use human and machine? double coverage eh?

sondraj
07-12-2007, 08:18 AM
Oh my God, another troll (sondraj). Federer a drama queen? You sure you know your definitions??

Yes, do you need me to break out the dictionary for the word drama. Many people have complained about hawke eye from day one calling it inaccurate, but now Fed makes a big stink about it during the final almost shedding tears because of it and now it's a hot topic, and now everyone wants to jump on the destroy hawk eye bandwagon.

lecter255
07-12-2007, 08:24 AM
i think they should use mac cam. they had it at US Open a few years ago, and it's a high speed camera recording the actual landing of the ball on the baseline. the picture is a bit fuzzy and grainy, but nothing is calculated. big events should combine that with the umpire's judgement.

mileslong
07-12-2007, 09:09 AM
I don't, for example, think that a ball should be called "in by 1mm" as happened in the final at Wimbledon. This means that there is a 39% chance the ball was actually out. The umpire is only supposed to overrule a line judge if there has been "a clear error." A 60% chance of an error is not clearly an error in anybody's language. Why should hawkeye be allowed to overrule a line judge without the kind of certainty an umpire has to have? I would suggest the reason hawkeye does not have a "don't know" option is that people are afraid that it will not be trusted if it admits to being imperfect. Are audiences really that immature that they are willing to believe something that pretends to know everything rather than something that admits what it does and does not know.
this is what i have been saying on these boards about this topic. my point has been that if a ball is so close to the line that the human eye cant really tell if its in or out then that also falls well within the error margin for hawkeye. if a balls lands far enough out that it lands outside of hawkeyes margin for error then the human can and usually does see it and they make the correct call anyway. ergo, on a ball like the one in the wimbly final NO ONE knows if it was really in or out because it falls well within hawkeyes margin for error.

the only way technology can really be around 100% is with that mac cam system which shows actual video footage in slow mo using high speed cameras. until then, its really just a publicity stunt...

Nadal_Freak
07-12-2007, 09:20 AM
O.K so fed complains about one stupid call and now hawke eye is broken. He is a drama queen and so is the tennis world for following in his footsteps
I agree. Fed needs to keep his mouth shut. Blaming the hawk eye is just pitiful.