Reverse engineering the USTA ratings algorithm. (NTRP, Tennis Link, League, Dynamic)

Discussion in 'Adult League & Tournament Talk' started by dizzlmcwizzl, Jun 27, 2013.

  1. schmke

    schmke Professional

    Joined:
    Jan 20, 2010
    Messages:
    1,320
    If my guess at who you are is correct, TLS lists you at 3.56 as of 11/18/12 and I have you at 3.74 at the same point, 0.18 higher. I'm guessing the 0.22 TLS is off is in the same direction so I'd be just 0.04 off, a lot closer.
     
    #51
  2. SunshineJS

    SunshineJS New User

    Joined:
    Mar 7, 2013
    Messages:
    44
    Well played
     
    #52
  3. dizzlmcwizzl

    dizzlmcwizzl Hall of Fame

    Joined:
    Mar 24, 2010
    Messages:
    2,176
    Location:
    DE
    More random thoughts as I work through the math of it all ....

    1st, TLS is limited in that it gives you a rating for a single district rather than combining them.

    Recently TLS published revised mid year ratings in my section. When I performed the calculations using just the results it uses by district I was able to get a new rating very close to what they had for a mid year rating. I did this calculation for all of my districts and for several of my friends.

    So it seems to me I could duplicate the TLS site fairly accurately, but of course that is not really my intention.

    Also, I understand now why TLS organizes it by district and does not combine them ... I believe it is just a function of how you get the data out of tennis link. I am sure they could code in a way to combine the results but probably is not worth it to them.

    Finally, I believe that TLS used last years year end ratings as a starting point for this year ... I have found that my attempts at performing calculations seem to zero in on a value as the data set increases. This leads me to believe that if the the TLS folks persist in this endevour they will become more accurate.

    By extension I suspect that for last years calculations they started everybody at the middle of their band (ie ... all 4.0's started at 3.75). This must be why the ratings became qualitatively much more accurate for folks that played lots of matches.
     
    #53
  4. dcdoorknob

    dcdoorknob Hall of Fame

    Joined:
    Mar 14, 2010
    Messages:
    3,560
    I just wish TLS wouldn't wait to do the southern section until dead last every time. :(
     
    #54
  5. dizzlmcwizzl

    dizzlmcwizzl Hall of Fame

    Joined:
    Mar 24, 2010
    Messages:
    2,176
    Location:
    DE
    Another set of random observations

    I have now played 18 matches this season ... I would like to share 2 observations.


    FIRST: I have gone through and performed calculations and it is apparent that big results create a huge variance in your rating.

    For example I have two results where I should have lost by 8 games or more yet won the match. I have another 2 matches where I should have won by 4 games or more but lost the match by 4+ games.

    So for the two big wins my rating prior to the match went up by almost 2 tenths after the match and for the big losses down by nearly 2 tenths.

    My belief is that this is to much volatility in my calculations. I have been using the max rating of the previous three ratings to calculate my running average, as described in an earlier post, but that provides to much upwards push on ratings.

    Second observation: If your goal is to elevate your rating you should have your biggest result a couple matches before the end of the season. BY getting the big result say in your 3 to last match you get to count that rating 3 or 4 times in your running average ... this means that even if you have expected results in your last three matches they will be perpetually average with the best of the last three and your big match will get folded in over, and over, and over ....

    The effect is less dramatic if you want to drive down your rating because a poor result may only be averaged once .....
     
    #55
  6. schmke

    schmke Professional

    Joined:
    Jan 20, 2010
    Messages:
    1,320
    Regarding cross district/section, I did it by district alone at first, but now do it nationally. Someone that plays in multiple district or even sections gets one rating from my algorithm. This improves general accuracy and has the added benefit of getting different districts/sections somewhat connected so they can be more accurate relative to each other.

    I also go back several years, not just one, to arrive at an accurate starting rating for this year. I've found this improves accuracy significantly.
     
    Last edited: Jul 14, 2013
    #56
  7. damazing

    damazing Rookie

    Joined:
    May 22, 2009
    Messages:
    251
    Off Topic: Nice job on knocking the Frankenteam out of playoff contention!
     
    #57
  8. dizzlmcwizzl

    dizzlmcwizzl Hall of Fame

    Joined:
    Mar 24, 2010
    Messages:
    2,176
    Location:
    DE
    They are still in .... all this means is that they will have to play us in the first round instead of getting a bye.
     
    #58
  9. dizzlmcwizzl

    dizzlmcwizzl Hall of Fame

    Joined:
    Mar 24, 2010
    Messages:
    2,176
    Location:
    DE
    Oh ... and thanks ... even tainted it was a nice team win
     
    #59
  10. Playtennis

    Playtennis New User

    Joined:
    Jun 29, 2012
    Messages:
    81
    Just looked at those TLS ratings. Question- what if you play in different leagues- do you average the TLS of all the leagues and is that your rating? When do next ones come out? Sept?
     
    #60
  11. dizzlmcwizzl

    dizzlmcwizzl Hall of Fame

    Joined:
    Mar 24, 2010
    Messages:
    2,176
    Location:
    DE
    I have not been attempting to decipher the TLS ratings, I have just been using them as a starting point on my journey of discovery for the USTA.

    On my latest iteration I used a weighted average of the matches in different sections ... for example, my personal rating that was just released had 6 matches in my home division and 4 matches in a neighboring district. When I calculated my personal average I weighted my home district 1.5 times my away district.

    Now, this has no basis in reality with the USTA ... it is just for me. When their next iteration comes out is anyone's guess ... probably December - January.
     
    #61
  12. dizzlmcwizzl

    dizzlmcwizzl Hall of Fame

    Joined:
    Mar 24, 2010
    Messages:
    2,176
    Location:
    DE
    We did not knock them out ... we just beat them in their last two regular season matches (against their C team) to keep them from getting a bye.

    However, they came out yesterday with their B+ team and gave us a shellacking. The championship score have not been posted yet, but I suspect they won again today.

    Their best player was in Texas trying to qualify in another section ....
     
    #62
  13. storypeddler

    storypeddler Semi-Pro

    Joined:
    Jun 22, 2005
    Messages:
    565
    Location:
    Hickory, NC
    Sigh. I wish the USTA would make such garbage illegal.
     
    #63
  14. mikeler

    mikeler G.O.A.T.

    Joined:
    Sep 26, 2008
    Messages:
    19,701
    Location:
    Central Florida
    The latest TLS ratings don't seem to be handlng the 4.5 40+ leagues very well in my area. I passed them around to various team member and we all got a good laugh when our 5.0 was lower than most of our 4.5s.
     
    #64
  15. HoustonHacker

    HoustonHacker New User

    Joined:
    Apr 19, 2007
    Messages:
    33
    Who and where? Will have to look him up...
     
    #65
  16. dizzlmcwizzl

    dizzlmcwizzl Hall of Fame

    Joined:
    Mar 24, 2010
    Messages:
    2,176
    Location:
    DE
    ... despite my belief that they are a better team than the one who won Nationals a couple years ago they lost in the championship today.

    Maybe it was us wearing them out the day before ... not likely.

    However the team they were playing against has three solid courts and got the perfect match ups.
     
    Last edited: Jul 22, 2013
    #66
  17. schmke

    schmke Professional

    Joined:
    Jan 20, 2010
    Messages:
    1,320
    "That's why they play the game".

    The "best" team doesn't always win, and match ups are a key reason why. I know I'm stating the obvious for many here, but you only have to win 3 courts to get the team win, and if you flip singles courts, or sacrifice court 1 doubles, and then get a favorable style match-up on another court, all of a sudden an upset happens.
     
    #67
  18. dizzlmcwizzl

    dizzlmcwizzl Hall of Fame

    Joined:
    Mar 24, 2010
    Messages:
    2,176
    Location:
    DE
    What is very funny is that the captain of the franken-team has a history of being random with his lineup. The captain of the weaker team is an exceptionally honest guy that never jerks his lineup.

    It just so happens the other guy randomized into the worst matchups he could hope for.

    ...... Karma
     
    #68
  19. schmke

    schmke Professional

    Joined:
    Jan 20, 2010
    Messages:
    1,320
    I'm a believer that if you are a strong team, playing with line-ups can only serve to hurt you. The only time you may benefit from it is if you are the "worse" team trying to pull a big upset.

    Two years ago I was on a 3.5 team that pretty much played it straight up every match and we advanced to Nationals and got 4th. This year I was on a 4.0 team that was good but not great and we played it straight up in local playoffs and got beat. In hindsight, our only chance would have been to flip some courts in the hopes we could pull and upset but straight up we lost 4-1.
     
    #69
  20. J_R_B

    J_R_B Hall of Fame

    Joined:
    Jan 27, 2010
    Messages:
    2,640
    Location:
    Newtown, PA
    I generally like to play straight up. The only time I don't is when I'm facing a captain who is a known stacker and I'm trying to unstack his lineup. I did that perfectly in 40s this year. We had one other top team in our league. In the first match, I figured he would stack and my partner wasn't available, so I put myself with our sub at #1 and we caught a team with a sub who was clearly the weakest doubles player in either lineup and we got a crucial point while our true #1 team beat their true #1 team at 2 in a 3-2 win. In the rematch, I figured they'd play straight since stacking didn't work the first time, so I just lined up straight and our true #1 team destroyed their true #1 at 1 and me and my partner eeked out a 3 setter at 2 in another 3-2 win.

    Since I had a good team this year, I saw a lot of stacking against us, but I usually figure that if a team is willing to just give us a court, why not take it and put them behind the 8 ball from the start?
     
    #70
  21. J_R_B

    J_R_B Hall of Fame

    Joined:
    Jan 27, 2010
    Messages:
    2,640
    Location:
    Newtown, PA
    I don't know all of the DE 4.5s, but the team that won has some pretty good players, too. I don't think that match should have been a slam dunk for the frankenteam. It's actually a flaw with the "frankenteam" approach. You end up with a bunch of guys who are great, but whose true loyalties really lie with other teams. These teams usually underachieve at some point.
     
    #71
  22. OrangePower

    OrangePower Hall of Fame

    Joined:
    Sep 7, 2007
    Messages:
    4,059
    Location:
    NorCal Bay Area
    Off topic, but nothing wrong with randomizing the lineup. I don't see this as being any less "honest" than playing straight up. The USTA makes it very clear that lines do not need to be placed in order of strength or in any order at all. I personally like to randomize my lineup, for several reasons:
    1. It saves me having to think about matchups. Really. Some captains spend a lot of time pouring over opponent's historical lineups looking for a possible matchup edge. I don't. :)
    2. It saves me from having to decide which of my singles players / dubs pairs are the strongest. Often it's too close to call. And also I don't have to explain to my guys why I think A is stronger than B.
    3. It means our opponents can't predict our lineup, since we have no clue ourselves!
     
    #72
  23. schmke

    schmke Professional

    Joined:
    Jan 20, 2010
    Messages:
    1,320
    This makes sense. Part of the reason why people who use my team reports have had success is because an upcoming opponent has been somewhat predictable in who plays which court which allows arranging a line-up that is as favorable as possible.
     
    #73
  24. dizzlmcwizzl

    dizzlmcwizzl Hall of Fame

    Joined:
    Mar 24, 2010
    Messages:
    2,176
    Location:
    DE
    To be clear ... I was not claiming the franken team manager was dishonest ... just that the other guy may be the most honest / genuinely good guy I know in tennis.

    The winning captain is a 5.0 + player guy that is captaining a team for the first time because no one else on the team would. He is an excellent 5.0 that cannot play for the team but he is good friends with those guys so he volunteered.

    The franken captain, after he won the Nationals in 2010, was quoted in the online USTA magazine that he uses a randomizing AP for determining the lineup.
     
    #74
  25. dizzlmcwizzl

    dizzlmcwizzl Hall of Fame

    Joined:
    Mar 24, 2010
    Messages:
    2,176
    Location:
    DE
    The winning team has a solid 8 players .. only 6 of whom were available yesterday. Those 6 are good but all but one are 45+ (most are 55+). The august heat may actually kill one of them if they play more than once a day.

    The franken team has 20 or more players that would have been in the in the top eight for the other team.

    As it turns out, the franken team would have been a force in Sectionals ... good players for sure but they have exceptional depth ... most in their 20's.

    No one will want to play our team in the first match of the weekend but by the afternoon match on Friday they will be spent ... they have no shot on the weekend, just because they simply do not have enough youngins ....

    Their second singles guy from yesterday was the guy I posted about beating 1 and 2 a couple months ago .... If I can beat him, and he makes their lineup they are in real trouble.
     
    #75
  26. dcdoorknob

    dcdoorknob Hall of Fame

    Joined:
    Mar 14, 2010
    Messages:
    3,560
    To me this is like a rock paper scissors situation. If you want to make sure you don't give up an edge to anyone, you can randomize (1/3rd each) what you throw, and no opponent will be able to have any sort of long term edge (non-luck or variance related) edge on you. It's unexploitable. The flip side is that this strategy will also not give you any long term edge on anyone else either. This seems like it is what this captain has chosen to do, and I personally respect the strategy choice as a normal mode of operation.

    The other captain who is predictable with his lineup is actually quite exploitable, but a captain who is set on randomizing his lineup against him won't be able to exploit it at all (just as randomizing 1/3rd each between rock/paper/scissors won't be able to exploit the opponent who always throws rock). Against this particular predictable captain, randomizing his lineup was likely a strategic mistake.

    Sorry, it's a bit of basic game theory stuff, something I'm far from an expert on but has been interesting to me.
     
    #76
  27. dizzlmcwizzl

    dizzlmcwizzl Hall of Fame

    Joined:
    Mar 24, 2010
    Messages:
    2,176
    Location:
    DE
    For that captain, in that year he had an exceptional team and could afford to let the chips fall randomly. Going completely random in your lineup will simply allow your team over the long run to win as many matches as they should .... no more, no less.


    However, perhaps the most winning situation is to go random when you have no clue what your opponents will do, and to stack when you are sure you know what is coming.
     
    #77
  28. dcdoorknob

    dcdoorknob Hall of Fame

    Joined:
    Mar 14, 2010
    Messages:
    3,560
    Yeah, I'd agree basically. If your opponent is predictable (exploitable), your best strategic choice is to go ahead and exploit them. Of course, just by doing this you are making yourself exploitable if the other team is able to anticipate correctly and adjust, but if your best read is that they won't, then it's probably the best strategic course of action.

    If you aren't sure what the opponent will do, I'd say randomizing the lineup is a sound strategic choice. You don't know how to 'exploit' them as they are unpredictable to you, so randomizing will ensure that they also can't exploit you.

    All this is theoretical, I'm not even a captain myself, but like I said it's just an application of game theory which is an interesting topic to me.
     
    #78

Share This Page