Question about USTA rating algorithm

Discussion in 'Adult League & Tournament Talk' started by storypeddler, Jun 1, 2013.

  1. storypeddler

    storypeddler Semi-Pro

    Joined:
    Jun 22, 2005
    Messages:
    565
    Location:
    Hickory, NC
    Let me see if I understand how this works, and if I'm wrong, would someone please correct me?

    If my doubles partner and I play another team that, according to the computer, we are supposed to beat 6-3, 6-3. and we do, in fact, beat them 6-3, 6-3, then we should not expect our dynamic rating to go up from that result, nor their dynamic rating to go down? The result was the expected result, so the numbers all remain as they were going in?

    Would it make any difference if you played a half dozen matches against different opponents, and in each case, won by the expected margin? No change in dynamic rating still?
     
    #1
  2. andfor

    andfor Hall of Fame

    Joined:
    Mar 16, 2004
    Messages:
    4,883
    Does anyone actually know the algorithm?

    What happens when your opponents play new opponents next week?

    Don't take this the wrong way, but what's the point trying to figure out the unknown?
     
    #2
  3. dizzlmcwizzl

    dizzlmcwizzl Hall of Fame

    Joined:
    Mar 24, 2010
    Messages:
    2,178
    Location:
    DE
    That is our understanding yes. If you keep winning (or losing) by the expected margin your rating will not change.

    Furthermore, if you only play with the same partner exclusively then your relative dynamic ratings difference will remain the same after each match.

    What I mean by this is that if you are a 3.60 and he is a 3.70 at the start of the season and you only play with each other then by the end of the season his dynamic rating will still be 0.1 higher than yours.... no matter what happens during the season.

    Further - further more, the USTA ratings system is supposed to be a zero sum calculation .... meaning if you and your partner do better than expected and after a match each of your ratings went up 0.05 points then the ratings of each of your opponents would go down by the exact same margin.
     
    Last edited: Jun 2, 2013
    #3
  4. schmke

    schmke Professional

    Joined:
    Jan 20, 2010
    Messages:
    1,369
    The USTA does not disclose the exact algorithm, but does offer quite a bit of information on how it works in various documents and FAQs that you can find on the USTA web-site(s) or by Googling around.

    And dizzy is generally correct about always playing with the same partner preserving the gap between the partners no matter what the scores are. But if one partner plays even one match by themselves or with a different partner, this isn't true as that would likely change their rating and one of the past 3 ratings that is part of the average is now different between the partners so the gap between them would change.

    To andfor's question, what your opponents from this week do the following week has no impact on your rating unless you happen to play them again the future or they become a benchmark player which would mean year-end benchmark calculations will factor in for your match against them at the end of the year.

    See my blog at http://computerratings.blogspot.com/search/label/tennis for some posts on this subject and a description of the Estimated Dynamic NTRP reports I can generate.
     
    #4
  5. tennixpl

    tennixpl Rookie

    Joined:
    Apr 22, 2013
    Messages:
    277
    Location:
    SA Texas
    ranting but in college football when number 2 loses to number 1 team why does the number 2 team lose ranking, this confirms the algorithm, its only when you lose as number 2 to number 3 should you be downgraded to some mystic level.

    its more amazing that the computer says you should beat them 6-3, 6-3. while it sounds dynamic i doubt it really it is it is probably just a fancy highly complex algorithm that is still static in nature, hey statisticians need jobs too.

    its all a guess and roll with the system, or get a stats degree and offer your services for free to the UTSA :)
     
    #5
  6. schmke

    schmke Professional

    Joined:
    Jan 20, 2010
    Messages:
    1,369
    This is an issue with pollsters than a properly written rating system. Some, but not all rating systems for football, mine included, look at the expected result and move a team according to how they do in relation to that, not necessarily just at win/loss. With such a system it is possible to lose a game and have one's rating increase, and corresponding one can also win a game and have the rating decrease.

    The NTRP rating system is indeed dynamic. Not sure what you are saying regarding it being complex but static in nature. The high level algorithm used is described in a variety of USTA documents so it isn't a real secret. They don't disclose many of the details of course so it isn't reasonably possible to exactly replicate their calculations, but I've come up with pretty good estimates. See the link in my other post for more info.
     
    #6
  7. Orange

    Orange Rookie

    Joined:
    Sep 14, 2011
    Messages:
    209
    It is difficult to compare football ratings to tennis ratings, in part because a football team has so many more players than a doubles tennis team. If the starting quarterback is injured and misses a game, the game is played anyway. Someone who knows this can take it into account for rating purposes. If a tennis player has an injury that is bad enough for him to miss a game, that game is not played.
     
    #7
  8. tennixpl

    tennixpl Rookie

    Joined:
    Apr 22, 2013
    Messages:
    277
    Location:
    SA Texas
    Orange- the point about football wasn't in respect to the way things are rated simply the concept of a number 2 losing to a number 1, shouldn't affect number 2 per se. they were supposed to lose and did so they might still be number 2 team, but in some rankings that means they drop several slots. schmke's point about the other non-numerical analysis that goes into those rankings make that happen, though in UTSA case i guess it doesn't .

    as far as team sports go in baseball they do know how an absence will affect the team, at least they have quantified it statistically, NBA too. i don't know if football has caught up on those stats methods yet. a lot more moving parts in football.

    schmke - i have looked at your page actually, it looks like a very intriguing approach. I'll look more when i get more time.
    My point about dynamic is that if i have regularly been raising in the rating do they take that trend into affect in my next match, or simple a static comparison of past matches weighted a certain way, similarly with who i beat last time, is he hitting his predictions or below or above them, and if so does that get fed back into what "should" have been our score and change the dynamic where i am at any given week.

    last week i played a guy and the score was supposed to be 7-6, 7-6 him. but i win 6-4, 6-4. now the next week his match he is supposed to win 6-2, 6-2 but he only wins 7-6, 6-4. bc hes injured. does the fact that a trend of this guy not doing as good as he "should" get fed back into my rating? that would be dynamic in my mind and i doubt that is the way it is done. Dynamic seems to mean the update it every match and go to a hundredth of a point. so one week i am a 3.15 and the next i am a 3.21. A truly dynamic system has a strong feedback loop

    just a question- if the prediction was wrong about a particular match up one the first match does it take that into account if i play that exact same player again in match two? perhaps against a particular opponent we have that whole match up thing where he just can't cope with one thing i do while other players can?


    i also hate secretive approaches bc it breeds contempt, i mean if people are going to game the system then they will game system, ppl can throw a match or ppl can throw games to make a rating what it is so, why does the UTSA hide it at all?
     
    #8
  9. cneblett

    cneblett Rookie

    Joined:
    Nov 12, 2009
    Messages:
    141
    I was in a captains meeting a couple of years ago and they provided a basic of show the calculation works.

    If you play singles, your full Usta rating, ex 4.06 is compares to your opponents rating, ex 4.18. They consider every .06 of difference to equal s break of serve and compare actual result to this. So in the example, the result should be, 6-4,6-4. If you win 7-6,7-6, they take the result number of breaks against expected. In this case, the difference is .12. They will take this number and divide by 2, and adjust both individuals by this amount. So after match, your new rating is 4.12and your opponents is now 4.12.

    In doubles they add your rating and partners rating together for your team and do same for opponents. They then do the same as for singles with expected score and result. Both partners will adjust by the same amount so the difference between you and your partner will be the same after match as before.

    Now how the other items like benchmark, your actual full rating, anything with tourneys, mixed, etc in the mix.

    Hope this helps.
     
    #9
  10. schmke

    schmke Professional

    Joined:
    Jan 20, 2010
    Messages:
    1,369
    Yep, this is consistent with what you'll find in the various USTA documents and FAQs. However, the rating you are saying is your "new rating" is actually the rating for that match which is then averaged with prior dynamic ratings. Perhaps that is what you are referring to as your "actual full rating".
     
    #10
  11. schmke

    schmke Professional

    Joined:
    Jan 20, 2010
    Messages:
    1,369
    Understand you now, and no, the dynamic rating isn't a "truly dynamic system" by your definition then. However, year-end ratings do have feedback loops at least for benchmark players for those that played them and these calculations probably do fit your definition. And not to drag us back to football but my football ratings do have a constant/iterative feedback loop.

    I don't believe there is any "match-up" component to the dynamic ratings. It simply generates a rating that reflects how you performed against your competition using the match scores the opponent(s) (and partners if applicable) rating at the time of the match.
     
    #11
  12. cneblett

    cneblett Rookie

    Joined:
    Nov 12, 2009
    Messages:
    141
    What we were told is that they keep your rating on a running full time. So there is not an averaging, they update it after each match played when the computer runs.
     
    #12
  13. schmke

    schmke Professional

    Joined:
    Jan 20, 2010
    Messages:
    1,369
    And that update is done by doing calculations, one of which is performing an average.
     
    #13
  14. bruce nissenbaum

    bruce nissenbaum Rookie

    Joined:
    Feb 18, 2004
    Messages:
    289
    As I learned from captain's meeting, DNTRP is essentially a simple arithmetic moving average of 4 consecutive matches: The latest calculated match rating is averaged with the previous 3. This lessens the impact of a single match and reduces volatility of the rating.
     
    #14
  15. Nostradamus

    Nostradamus G.O.A.T.

    Joined:
    Nov 30, 2011
    Messages:
    13,233
    Location:
    In the future
    I just wish you know where you are at all times. I don't think that will lead to cheating. like if you are at 4.8, I like to know that.
     
    #15
  16. dizzlmcwizzl

    dizzlmcwizzl Hall of Fame

    Joined:
    Mar 24, 2010
    Messages:
    2,178
    Location:
    DE
    I think most folks would like to know ... and only a few would use it for evil.

    However, the USTA's ultimate goal is to collect as much in league fees as possible. I am not sure this change would drive players towards paying more league fees. Players would be ambivalent at best, manipulative at worst and players at the low end of the band might simply not get to play.
     
    #16
  17. Nostradamus

    Nostradamus G.O.A.T.

    Joined:
    Nov 30, 2011
    Messages:
    13,233
    Location:
    In the future
    What if USTA charged a fee every time you want to check your dynamic rating ? Lets just say, they charge $40 each time you want to check your rating during the season. That could be a money maker for USTA.:)
     
    #17
  18. schmke

    schmke Professional

    Joined:
    Jan 20, 2010
    Messages:
    1,369
    I'll do it for less :)
     
    #18
  19. Bash and Crash

    Bash and Crash Semi-Pro

    Joined:
    May 8, 2008
    Messages:
    564
    I know my team captains were happy with your results in sectionals. Glad to see my rating on your system is comfortably under 4.5 so no bump up yet.
     
    #19
  20. OrangePower

    OrangePower Hall of Fame

    Joined:
    Sep 7, 2007
    Messages:
    4,071
    Location:
    NorCal Bay Area
    I think USTA publishing a more precise rating would be a very bad idea.

    It will lead to some cheating, by allowing people to more effectively "manage" their rating so that they don't get bumped up.

    But that would not even be my main concern. More concerning is that people are going to view the rating as an absolute measure of ability, when in fact it is not nearly that precise at the micro level.

    For example, imagine a captain puts Player A in the lineup for a crucial match instead of Player B, despite the fact that Player B has a 0.1 higher rating. The captain could very well think A is actually a better player, or a better matchup for the potential opponent. Now a good teammate would accept this, but as we know there are always squeaky wheels, and this would just give them more ammo.
     
    #20
  21. andfor

    andfor Hall of Fame

    Joined:
    Mar 16, 2004
    Messages:
    4,883
    Given the way the current USTA NTRP system works I have to agree. I do like how rating systems in other countries work and show the players ratings with a little more detail. But in many countries tennis ratings are made up mostly from tournament results. Until the USTA places more emphasis on tournament play for ratings which would curb sandbagging and tanking, it's better left alone. Overall our system is not that bad, it's easy to signup and play without much fuss. At the end of the day that's really what's best. For those that want to avoid the drama of NTRP team tennis play a tournament.
     
    #21
  22. schmke

    schmke Professional

    Joined:
    Jan 20, 2010
    Messages:
    1,369
    I think this is an interesting point to consider. On one hand, including tournaments, and even giving them more emphasis like you describe, would give sandbaggers more opportunity to tank matches to keep their rating down wouldn't it? For a sandbagger, how would this be different than say a Fall league that counts for rating purposes being used to tank matches?

    Yes, a tournament only gives you one chance to lose generally and not multiple matches that a league gives you, but there are many tournaments one could play if they are intent on losing a bunch of times.

    And from what I've seen, tournaments oftentimes draw out the player that doesn't play league tennis and so may not have much of a ratings history or be self-rated and thus their rating isn't that accurate and basing other player's ratings on their performance against them may be questionable.

    Of course, there is an argument for including all the results you can, and tournaments should be where someone is playing all out as it is one and done, but I think we all know things aren't always as they "should be".

    Note that some of the cons against including tournaments also apply to counting leagues other than the main ones leading to Nationals. For a sandbagger, these other leagues are just opportunities to game the system.
     
    #22
  23. OrangePower

    OrangePower Hall of Fame

    Joined:
    Sep 7, 2007
    Messages:
    4,071
    Location:
    NorCal Bay Area
    I would agree with your second point, that tournaments attract some players who don't play much league and thus could skew ratings.

    But I don't agree that tournaments are better for tanking, for a couple of reasons:

    1. League is a team thing, and so you can win the team match while tanking your individual match. And also sometimes there are meaningless team matches, for example if a team has already made playoffs, so good opportunity for tanking. On the other hand, tanking a match in a tournament is losing the tournament there and then.

    2. Since you pay entry fees per tournament, ratings management via tournament tanking is a costly proposition. Whereas league ends up being much cheaper per match (in most places; of course there are leagues that are played indoors and have much higher court costs per match, but that's the minority nationwide).
     
    #23
  24. dcdoorknob

    dcdoorknob Hall of Fame

    Joined:
    Mar 14, 2010
    Messages:
    3,560
    Yeah, a league match here typically costs $3 per person for court fees, and like $30 for a whole season on a team.

    Whereas a tournament can cost $35-40 for one entry.

    If guys want to pay up like that just to tank, whatever, let them, but certainly agree that league matches are much more economical for the prospective tanker or 'ratings manager'.
     
    #24
  25. andfor

    andfor Hall of Fame

    Joined:
    Mar 16, 2004
    Messages:
    4,883
    You hit the nail on the head. I don't pretend to have the perfect mousetrap, but if the USTA was more concerned about competitive level accuracy and less on general participation we'd see additional rating requirements. Like having to play X (insert # like 3) tournaments per year to keep/maintain their rating. If someone is a big enough jackass to pay for 3 tournaments and tank let them be. But that bring up more issues like time available to play the tournaments, hosting them, geography, etc.

    I say our system for the most part works. Oh well, time to go play tennis at my club league with buddies who don't care about ratings.

    Happy Father's Day!
     
    Last edited: Jun 16, 2013
    #25
  26. SunshineJS

    SunshineJS New User

    Joined:
    Mar 7, 2013
    Messages:
    44
    Here's the proof that people will try and game the system and tank tournament matches... USTA #377843

    Search that number and look at his record from 2012... 22-2 in league play, with usual convincing wins and 1-6 in tournaments with several 0-0 losses!

    Tournaments count towards ratings in Colorado and this is the argument for why Southern and others do not count them.
     
    #26
  27. OrangePower

    OrangePower Hall of Fame

    Joined:
    Sep 7, 2007
    Messages:
    4,071
    Location:
    NorCal Bay Area
    Out of interest I looked at this guy's record.

    Maybe you have inside info, but to me it's not at all conclusive that he was intentionally tanking tournaments to manage ratings.

    Yes, he was 22-2 in league (actually I saw 4 league losses but whatever), and only 1-6 in tournaments, but his league play was all dubs, whereas 4 of his 6 tournament losses were singles. And the guy is a super-senior, so it's completely understandable if his singles play is not at the level of his dubs play. He probably has no opportunity to play singles in league and so can only scratch that itch in tournaments.

    Yes I know that ratings management happens, but I think people jump to that conclusion too quickly at times.
     
    #27
  28. SunshineJS

    SunshineJS New User

    Joined:
    Mar 7, 2013
    Messages:
    44
    You're right I do have inside info... but if you just watched him play it's an obvious tank as well. He doesn't even move for returns and will be down Love-40 and lob a serve in then immediately walk to the post to start changing the score without even waiting for the return to come back. Two of those 0-0 matches were done in all of 20 mins. Plus here's an article from years ago where the guy admits it... http://www.westword.com/1999-09-09/news/an-ugly-racket/
     
    #28
  29. OrangePower

    OrangePower Hall of Fame

    Joined:
    Sep 7, 2007
    Messages:
    4,071
    Location:
    NorCal Bay Area
    Wow! Good read - thanks for the link. Sad that he would still be doing this kind of stuff years later as a super-senior. You'd think he'd be older and wiser by now.
     
    #29
  30. robert

    robert Rookie

    Joined:
    Aug 9, 2009
    Messages:
    106
    USTA rating can't handle two players who are more than 0.5 apart from each other. That is why they only allow a player to playup one level in league. But in tournament, it could be 1.0 different. So it doesn't work.

    E.g. a 4.0 players open tournament against 5.0 player. Event 6:0;6:1 will bump up the 4.0 player's rating and bump down 5.0 player's rating.
     
    #30
  31. NumbersGuy

    NumbersGuy Rookie

    Joined:
    Jun 5, 2012
    Messages:
    153
    Correct. That's why the algorithm disregards matches where the players are more than 0.5 apart.
     
    #31
  32. cneblett

    cneblett Rookie

    Joined:
    Nov 12, 2009
    Messages:
    141
    Wonder if that will apply to the plus leagues this year? A 4.0 vs a 5.0 in the over 40 4.5 for example. I had one of those couple weekends back and everyone played legit and result was the 2 5.0's won 3,3 over a 4.0 and a 4.5.
     
    #32
  33. schmke

    schmke Professional

    Joined:
    Jan 20, 2010
    Messages:
    1,369
    I don't know that the USTA completely disregards matches if the ratings are 0.5 or more apart. In the scenario you describe, one might argue that the 4.0/4.5 played it closer than expected so their rating should improve (and the 5.0s ratings come down) rather than just ignoring the match.

    On the other hand, one might argue this is unfair to the 5.0s, essentially putting them in matches where they have to win 0 & 0 or risk their rating going down. See this for a discussion on the side-effect of the plus leagues on the 5.0s (and 5.5s) ratings. http://computerratings.blogspot.com/2013/06/how-usta-is-inadvertently-bumping-50.html
     
    #33
  34. Gut4Tennis

    Gut4Tennis Hall of Fame

    Joined:
    Apr 2, 2013
    Messages:
    2,318
    Location:
    Tennis Court
    jaw on floor
     
    #34
  35. storypeddler

    storypeddler Semi-Pro

    Joined:
    Jun 22, 2005
    Messages:
    565
    Location:
    Hickory, NC
    I'm guessing he must have a very small..."racquet". Hard to come up with any other reason for this kind of over-compensating. He seems to be desperately trying to feed his ego by captaining these teams and is so arrogant that he basically shoves it in the face of the USTA. His last comment about being able to get any player he wants in the whole state tells you all you need to know about someone like this. At some point, the USTA should simply bar him from league play in any role. What a sad human being.
     
    #35
  36. doubleshack

    doubleshack New User

    Joined:
    Dec 15, 2009
    Messages:
    84
    What if, in addition to more precise ratings, they ranked people.

    First, I completely agree, at a micro level, ratings are meaningless. If someone is x.2, they are not necessarily better than someone at x.1. There are too many factors involved. Ratings at a macro level give a high probability of competitive matches. Making the ratings more precise will not improve that probability, simply because precision does not equal accuracy.

    Anyway, back to rankings. They would be meaningless for the same reason micro level is meaningless. However, they would have an ego effect. If you went to the USTA website and saw that you were play #1024 at level x.5, wouldn't you want to raise your ranking, even though you know it is not accurate? To that end, would it reduce sandbagging?

    I realize there will always be sandbaggers. However many of them have ego's, and while they don't mind throwing a match to save their rating, what if it also lowered their ranking? Would Sam the sandbagger still throw a match if that meant he would be lower ranked than his buddy Bob.

    We have a tencap rating system in our area. It is more granular than USTA. There are several other differences, but that's another thread. In short, your tencap rating is a bit of a badge of honor. I'm willing to say there is less sandbagging because of it.

    I wonder if we ranked people in the USTA based on league matches, would your ranking become a badge of honor, and reduce sandbagging.
     
    #36
  37. andfor

    andfor Hall of Fame

    Joined:
    Mar 16, 2004
    Messages:
    4,883
    I like the ideas of ranking players based on league wins, not showing the actual rating though. Unintended consequence would be players get bumped up ranked below other and another crying fest would start.

    Until the incentive for winning each individual match in team play is increased and the reward for losing (moving down in rating) hurts more the insanity of the minority who dream of "recreational league tennis greatness" will continue.

    Really have to think if there was an easy answer it would have been found by now.
     
    #37
  38. NLBwell

    NLBwell Legend

    Joined:
    Jun 18, 2004
    Messages:
    7,157
    I am the not necessarily so proud owner of a 1 and 0 win over said opponent many years ago. I had played only a few league or tournament matches in this area and had no idea of the intricacies of USTA league. I was expecting a tough match and couldn't figure out what was going on. After the match, a couple of people explained it to me. I was (and pretty much still am) amazed that anyone would go through the trouble of entering a tournament, driving across town, and then throwing a match like that.
    Maybe because I played at much higher levels where there was actually something riding on the matches (at least the possibility of a college scholarship) but what would the psychology of someone be to go through all of that for a meaningless recreational league trophy?
    I would think that the purpose of playing recreational leagues would be to challenge yourself at the highest level you could achieve, while still having fun, getting out of the house, and having some chips and beer.
     
    #38
  39. storypeddler

    storypeddler Semi-Pro

    Joined:
    Jun 22, 2005
    Messages:
    565
    Location:
    Hickory, NC
    It is, certainly, an interesting phenomenon. Admittedly, I am very competitive and love to win. I think a lot of tennis players fall into that category, and I don't apologize for it. But my intense desire to win doesn't override my ethics to the point where I am willing to call line-balls out to steal points. But not everyone has ethics, or at least not the same set of ethics.

    In the article (from the given hyperlink) about this guy, he freely admits recruiting players who are too good to play at a particular level and telling them to throw matches in order to cheat the system. He admits it and comes up with some lame excuse that he thinks the USTA system is wrong, so he just does whatever he feels like to sidestep it and get what he wants. The old line about the ends justifying the means, in other words. He clearly wanted the "glory" of going to nationals so badly that he was willing to cheat left and right AND instruct his players how to do the same in order to get what he wanted. He knows it is cheating and he simply doesn't care. This is a slimy individual IMO.

    But sadly, I don't think his type is all that rare. Tonya Harding. Lyle Alzado. Mark McGwire. Barry Bonds. Lance Armstrong. I guess all we can do is try to weed them out as we discover them and prevent them from ever coming near the sport again.

    Ah, the dark, seedy underbelly of human nature. Rather ugly.
     
    #39

Share This Page