2012 TLS Accuracy at your facility - TLS missed in 2012 at my facility

Discussion in 'Adult League & Tournament Talk' started by g4driver, Sep 19, 2013.

  1. g4driver

    g4driver Hall of Fame

    Joined:
    Sep 24, 2009
    Messages:
    1,720
    I started this thread after looking at the 2013 TLS estimated ratings for my local venue.

    I feel the TLS estimated ratings aren't very accurate compared to Kevin's for the men at my local club. I could be very wrong, and yes, I could have missed a guy or two and my numbers could be a little off. I am asking Kevin to look at them an make a comment or correction to my numbers. There were several 3.0S players who starting playing during Mixed or Combo and I left there data out intentionally as they would still be S rated or M rated starting Jan 2013.

    Disclaimer:

    I don't get any freebies, kickbacks or discounts from Kevin for making this post. I made this post to reflect my opinion that the TLS ratings miss the mark when predicated bump ups and bump downs for the men at my local facility.


    Let's look at the facts from the TLS 2012 Men's estimated ratings vs the actual 2012 USTA year end ratings.

    TLS seems to miss when predicting a bump down.

    0 / 7 on 4.0 Men predicted to be bumped down to 3.5
    0 / 10 on 3.5 Men predicted to be bumped down to 3.0

    That is 0 for 17 on Bump Downs at my club. 0/17 isn't very accurate IMO.

    Looking at bump ups, TLS is a little better

    0 / 1 on 4.0 Men bumped to 4.5 (he was bumped - TLS said no bump)
    0 / 3 on 3.5 Men bumped to 4.0 (three guys bumped - TLS said no bump)
    3 / 7 - on 3.0 Men bumped to 3.5 --

    TLS got 3 out of the 7 correct - The three out of five that TLS guessed would be bumped up (so three correct and two wrong guesses), plus two who weren't predicated to be bumped actually were (two more TLS misses). TLS forecasts two more bumps that didn't happen.

    3 out of 11 on bump ups.

    TLS correctly forecast three 3.0s would be bumped to 3.5.
    TLS incorrectly forecast bump ups or downs for 25 other men.

    It looks like TLS went 3 for 28 on it's forecast for bump up/downs for my club in 2012.

    If I missed a number or stat, please forgive me TLS. Not trying to slam you, but just giving you the facts after the official ratings were released.

    v/r
     
    #1
  2. Maui19

    Maui19 Hall of Fame

    Joined:
    Sep 18, 2010
    Messages:
    1,649
    I don't see how you can draw any conclusions about the accuracy of either Kevin's or TLS' DNTRP estimates when compared to USTA year-end ratings. Both Kevin's and TLS' numbers are mid year estimates, while the USTA's are year-end.
     
    #2
  3. gmatheis

    gmatheis Hall of Fame

    Joined:
    Aug 10, 2010
    Messages:
    1,553
    At least for southern only the spring season counts towards your rating, which means a mid year estimate would be based on all the same information that will be used for your year end rating.

    TLS has me rated at 3.29 in Greenville, SC even though I was 21-0 in my Greenville spring leagues.

    I'd love to be a 3.5 for one more year :twisted:
    But I doubt I will be.
     
    #3
  4. J0EBL0W

    J0EBL0W New User

    Joined:
    Nov 16, 2012
    Messages:
    42
    TLS said I'd move down and I didn't. I wasn't even able to appeal my rating.
     
    #4
  5. g4driver

    g4driver Hall of Fame

    Joined:
    Sep 24, 2009
    Messages:
    1,720
    My point is this: TLS made estimates. Ok so mid-year estimates in May of 2012, and evidently at my facility, TLS is so off the mark that it only got 3 out of 28 bump ups / down correct.

    TLS missed the 4.5 who was bumped by 0.14 - TLS guessed 3.86 - he was bumped to 4.5 not by appeal but by the USTA year end ratings.

    Of those 28 players I mention at my facility, only six were B rated players, meaning the other 22 did not advance past local playoffs. The only matches that affect anyone in the Southern Section are the matches played in the spring, local & State playoffs and beyond to Sectionals and Nationals. Since 22 of these 28 players didn't advance to state playoffs, those TLS estimates aren't going to get a lot of movement on their DNTRPs.

    Yes, I get that State/Sectional/National playoffs will affect everyone's DNTRP, but these 22 players were done minus the State/Sectional/National adjustments

    Do you really think a State/ Sectional /National adjustment is going to move a player by .14 or .20, or more? I don't

    Kevin's estimate for me (made after our state playoffs concluded) and the TLS estimate (made as of 8/22/2013) for me in 2013 are off by 0.22 That means all State playoffs and the Southern Sectionals were completed as of 8/22/2013

    TLS's mid-year 2012 estimate was wrong 3 out 28 times at my club. That seems like a miss to me.

    I drew the conclusion that I don't see 2012 TLS Estimate (as of 5/17/2012) to be very accurate at all. Time will tell for 2013.
     
    #5
  6. g4driver

    g4driver Hall of Fame

    Joined:
    Sep 24, 2009
    Messages:
    1,720
    TLS also has some of your teammates like BM rated 3.41 and staying at 3.5 - That's downright funny.
     
    #6
  7. Maui19

    Maui19 Hall of Fame

    Joined:
    Sep 18, 2010
    Messages:
    1,649
    Well according to Kevin's estimates, few if any of my teammates who went to sectionals look like they're getting bumped. So I don't know how accurate any of this stuff is.
     
    #7
  8. gmatheis

    gmatheis Hall of Fame

    Joined:
    Aug 10, 2010
    Messages:
    1,553
    haha I saw that ... I'm sure he would LOVE to be a 3.5 next year :p
     
    #8
  9. J_R_B

    J_R_B Hall of Fame

    Joined:
    Jan 27, 2010
    Messages:
    2,632
    Location:
    Newtown, PA
    Here are my comments about TLS:

    1. TLS can be useful as a tool to get relative rankings within a section/district. It is not useful as a tool to predict bumps.

    2. TLS was far less accurate in 2012 than 2013 relative to my local knowledge of actual player levels around here.

    3. Kevin's algorithm is far more accurate and sophisticated than TLS at replicating actual USTA DNTRP calculations, and it's not even close. It's not free, though. You get what you pay for.

    There's no need to bash TLS. They do what they do, and it's up to people to interpret it correctly, use it where it's useful, and ignore it where it's not. I'd certainly rather have their ratings out there than not. You can always just ignore it if you want.
     
    Last edited: Sep 19, 2013
    #9
  10. schmke

    schmke Professional

    Joined:
    Jan 20, 2010
    Messages:
    1,287
    What list are you working off of? None of the lists I've posted include sectional play, they were generated earlier in the year, so it is unlikely they'd be a good predictor of bumps for players that played in sectionals.
     
    #10
  11. Maui19

    Maui19 Hall of Fame

    Joined:
    Sep 18, 2010
    Messages:
    1,649
    True, but I am extrapolating. Our team members performed more or less as I did in sectionals (actually probably worse) and my rating plummeted as a result of sectionals. I would suspect everyone else's did too. Perhaps that's not a valid assumption.
     
    #11
  12. J_R_B

    J_R_B Hall of Fame

    Joined:
    Jan 27, 2010
    Messages:
    2,632
    Location:
    Newtown, PA
    It's doubtful anyone's rating "plummets" at sectionals. By that time, you're pretty much only playing others with a high DNTRP, so unless you're losing 0 & 1 every match or something, your rating can't be that low.
     
    #12
  13. g4driver

    g4driver Hall of Fame

    Joined:
    Sep 24, 2009
    Messages:
    1,720
    JRB, there was no bashing - just a posting of factual information. If you took that as bashing, so be it . The only conclusion I drew was TLS misses on 25 out of 28 forecasts for bumps.

    There was no intent to bash TLS.

    I hope the data from my facility and others like gmatheis might help TLS tweak their algorithm. If TLS looks back at their data, and compare it's estimates to the actual year end ratings, it would seem TLS could tweak their algorithm.

    I think gmatheis will agree, that several of his teammates, as a few of mine are very underrated by TLS.
     
    Last edited: Sep 20, 2013
    #13
  14. schmke

    schmke Professional

    Joined:
    Jan 20, 2010
    Messages:
    1,287
    And every match is specific to those involved and there is no general rule. One player may have bad matches against weak opponents and have their rating drop, but that doesn't mean their teammates will drop too even if they lost as it depends on the ratings of their specific opponents.
     
    #14
  15. schmke

    schmke Professional

    Joined:
    Jan 20, 2010
    Messages:
    1,287
    Also, FWIW, the reason for the "plummet" was the last two matches at Sectionals were a 6-0,6-2 loss in doubles and a 6-2,6-1 loss in singles. Even against strong opponents within the same level, these scores will likely drop one's rating.

    Now, if the opponents are sandbaggers that are underrated because they've been artificially keeping matches close to manage their rating, the drop in rating from these losses may be unfair, but the computer can't factor this in as part of the match by match dynamic ratings. It will get factored in as part of the year-end benchmark ratings though. If these opponents show themselves to be stronger through their sectional and perhaps national play, then these losses won't look as bad in the year end calculations.
     
    #15
  16. beernutz

    beernutz Hall of Fame

    Joined:
    Aug 16, 2005
    Messages:
    4,397
    Location:
    expanding my Ignore List
    How did TLS know what the starting dynamic rating was for each player at the beginning of 2012? Did they analyze all the available data for each player from the time they started entering data in tennislink or did they just start everyone at a halfway point between rating levels (e.g. 3.5 rated players were started at a 3.25 dynamic rating)?
     
    #16
  17. goober

    goober Legend

    Joined:
    Jun 9, 2004
    Messages:
    8,491
    I use TLS for a general guideline for how close somebody is to being bumped and relative standing of team members. For this purpose it is fine. I personally think that worrying about exact ratings is a fruitless endeavor.
     
    #17
  18. robert

    robert Rookie

    Joined:
    Aug 9, 2009
    Messages:
    106
    It is hard to say TLS vs Kevin. I feel it is 50 50 between them.

    There is only one way to verify who is better - year end bump up/down. But samples are small since most of players will stay at the same level for years.
     
    #18
  19. damazing

    damazing Rookie

    Joined:
    May 22, 2009
    Messages:
    246
    An interesting observation about the TLS stats - it gives a different rating by county for players that play in multiple counties. What it appears to do is to rank the players within the county for all the matches played in the county - an example of this is there is one county nearby that gets mostly players from the surrounding counties that play on teams. The highest rated person is close to the cutoff for 4.0 and almost at 4.5, but that would only work if he hadn't played any other county - in the three other counties he plays in he is ranked closer to the mid-point of the 4.0 rating.

    Also, based on a few losses, guys that are middle of the road to upper level 4.0's in the larger counties are showing up as 3.5's in this smaller county which would only work if the algorithm started them off at the bottom of the rating at the beginning of the season.
     
    #19
  20. J_R_B

    J_R_B Hall of Fame

    Joined:
    Jan 27, 2010
    Messages:
    2,632
    Location:
    Newtown, PA
    It's done by district. If your districts are organized by county, then it's by county for you. There are a lot of crossover players in NJ, too, between MS NJ, MS Philly, MS Delaware, and Eastern NJ. I personally have two different TLS ratings, one that says I should be bumped to 4.5 and one that says I'm still under the threshhold.
     
    #20
  21. schmke

    schmke Professional

    Joined:
    Jan 20, 2010
    Messages:
    1,287
    I did a report for someone in Mid-Atlantic that had 4 TLS ratings and they varied a full 0.22. And while I haven't checked the range on all of them, I've had a lot of customers in the Mid-Atlantic, Middle States, Eastern, and New England sections like this where it is easy to play in multiple districts/areas and they play in 3, 4, or more of them.

    The reports I generate incorporate a player's matches in all areas/districts/sections to create a single rating and so don't suffer from this shortcoming.
     
    #21
  22. J_R_B

    J_R_B Hall of Fame

    Joined:
    Jan 27, 2010
    Messages:
    2,632
    Location:
    Newtown, PA
    Mine is 3.86 in one district and 4.02 in the other, so a difference of 0.16.
     
    #22
  23. g4driver

    g4driver Hall of Fame

    Joined:
    Sep 24, 2009
    Messages:
    1,720
    TLS removed the link to their 2012 data from their homepage.

    From my perspective, their 2013 data at my club is even less accurate for several friends of mine. We'll see how close they are around the first of Dec.
     
    #23
  24. Baxter

    Baxter Professional

    Joined:
    Jun 18, 2008
    Messages:
    1,309
    Location:
    Colorado
    I noticed that and was wondering why. And who is Kevin?
     
    #24
  25. beernutz

    beernutz Hall of Fame

    Joined:
    Aug 16, 2005
    Messages:
    4,397
    Location:
    expanding my Ignore List
    shmke is Kevin.

    You have to wonder if someone from TLS is monitoring this forum and these TLS threads. Or it could just be a coinky-dink.
     
    #25
  26. NLBwell

    NLBwell Legend

    Joined:
    Jun 18, 2004
    Messages:
    7,068
    To me, TLS seems to be fairly accurate within the district. What it didn't seem to account for well was the fact that there were almost no USTA bump-ups this year in the section. The entire section seems to have been bumped down relative to other sections. Therefore, he predicted bump-ups that did not occur, but would have occurred in average years.
     
    #26
  27. schmke

    schmke Professional

    Joined:
    Jan 20, 2010
    Messages:
    1,287
    That would be me. I do estimated ratings and generate reports and periodically post lists. See http://computerratings.blogspot.com/search/label/tennis and http://sites.google.com/site/computerratings/usta-tennis

    It is a small sample, but in the comparisons I've done with TLS and observations others that I've done reports for have made, I've found my ratings to be more accurate in predicting bump ups/downs.
     
    #27
  28. Baxter

    Baxter Professional

    Joined:
    Jun 18, 2008
    Messages:
    1,309
    Location:
    Colorado
    I took a look at TLS and IMO it is spot on for my area. It found the one 4.5 playing 4.0, the three 3.5s who rated themselves 4.0 and are still there for some reason, and a few 3.0s that play up at 3.5. We're only talking about seven or eight people, but it got them all correct, at least in my opinion.
     
    #28
  29. J_R_B

    J_R_B Hall of Fame

    Joined:
    Jan 27, 2010
    Messages:
    2,632
    Location:
    Newtown, PA
    There are a bunch of different levels to the TLS debate. One is how accurately do they replicate the USTA DNTRP calculation? The answer to that is not always very well. But the other questions are how accurately do they and the USTA reflect actual relative skill levels? This is still an open question. It's a given that TLS significantly varies from USTA ratings, but it's not a given that that means they are worse at giving tennis players a relative ranking within a district. Kevin is definitely better at accurately mimicking USTA DNTRP ratings, but that's also his goal - to estimate actual DNTRP ratings. It's not 100% clear if that is the goal of TLS, or if it's just to give players the most accurate relative rank in a district, which the USTA DNTRP may or may not do.
     
    #29

Share This Page