TLS Rating vs Tennisrecords

J011yroger

Talk Tennis Guru
The more I know about UTR, the more obvious it is that the algorithm needs serious revision to stabilize ratings.

With the current UTR algorithm, if I were to beat a 12-year-old munchkin, and 5 years later that kid grows up to win Wimbledon. My UTR would get dragged up and say I’m as good as Roger.
They really need to address this if they want to be taken more seriously by everyone. They need to go to an ELO algorithm that takes a snapshot of opponent strength at the time the match was played, and then allows that a player’s level might go up or down over time.


From UTR's site (a FAQ about UTR):

"Why did my rating change when I haven’t played recently?
UTR is a measurement of player skill today and recalculates daily. There can be several reasons why a change (up or down) would occur. Some reasons could include:
  • You played a player that was unrated/projected and their rating has become more reliable through more play.
  • Scores from more than 12 months ago fell off your record"
And other places in that document also mention the ignoring of results from over one year old. I think it's been mentioned in another UTR thread.

Years-of-Academy-Training-Wasted-10102019150233.jpg


J
 

OnTheLine

Hall of Fame
UTR is really designed based off it seems the needs of junior players. So for a 14 year old they don't want their capabilities at 12 years old depressing their current ratings .... as adults, results 12 months old are probably more reflective of our current state of affairs than those who are quite young.

But what is miserable is having an excellent result against a higher rated player who then falls apart.

I am a whopping UTR 3.16 ... I beat a 4 by a nice 3&3 scoreline in singles. My UTR went up nicely.

About 2 weeks after that match my opponent retired from a match against another UTR4 injured. Lost that match 6-2; 3-0 .... I was idle ... my rating dropped. Opponent played another match coming back likely too soon from injury and lost (retired again) to a UTR2 6-4; 2-1 retired. my rating dropped even further. I only had mixed matches at the time that don't count.

So if you care about your UTR (which we should not) be certain to play only against healthy people, or at least people who are smart enough to fully recover before playing more matches.
 

travlerajm

Talk Tennis Guru
UTR is really designed based off it seems the needs of junior players. So for a 14 year old they don't want their capabilities at 12 years old depressing their current ratings .... as adults, results 12 months old are probably more reflective of our current state of affairs than those who are quite young.

But what is miserable is having an excellent result against a higher rated player who then falls apart.

I am a whopping UTR 3.16 ... I beat a 4 by a nice 3&3 scoreline in singles. My UTR went up nicely.

About 2 weeks after that match my opponent retired from a match against another UTR4 injured. Lost that match 6-2; 3-0 .... I was idle ... my rating dropped. Opponent played another match coming back likely too soon from injury and lost (retired again) to a UTR2 6-4; 2-1 retired. my rating dropped even further. I only had mixed matches at the time that don't count.

So if you care about your UTR (which we should not) be certain to play only against healthy people, or at least people who are smart enough to fully recover before playing more matches.
I think the UTR system has inherent downward drift (during periods of inactivity) for adult players, simply because they are being compared against juniors whose ratings are inherently drifting upward. If adult ratings didn’t drift downward inherently to create balance, the algorithm would collapse by spiraling upward for everyone!

I hope they fix this obvious problem soon.
 

pixel

New User
First, USTA does NOT consider tournaments in all regions/sections ... only in some. And from information from my LLC they count for much less than people think.

I am guessing you are joking about finding the secret ... :) I wish it was that simple. But if you lose 4 games to opponents that the computer thinks you should double-bagel ..... well, the "secret" won't work.

If I were to put out a "secret" it would be: In doubles, play with the lowest rated person you can find (say a 2.5 playing up on your 3.0 team), play on line 1 assuming you play against higher rated players and greatly beat what the computer thinks the score should be.

The REAL secret is: practice hard, become better, dominate your level and you will be bumped up.
:)
Yup, the secret is work hard on your game! Sometimes we take the ratings too seriously!
But you are right, play with lower rated players help !

Also on other topics discussed here;

If you are a USTA member, you have access to all their data and it only costs you $44 per year!
We had a presentation from USTA about the ratings right after Tennisrecord.com started. They were not happy about it and insisted that none of these sites are related to USTA. But they mentioned that the TLS is more accurate than tennisrecord.com
 

Elevenis

New User
Most of the 4.0 players in our league and are UTR 6 with a range from 5 to 7, so 4.5 at UTR 7.36 would be in the lower 4.5 possibly?

You might be right. I only just joined myUTR so, admittedly, I may not have browsed enough local players... although it seems very few have claimed their profiles that I know.
 

OnTheLine

Hall of Fame
Yes, please tell us how we can see their proprietary dynamic ratings. I'd love to see where I am at in relation to the third-party estimators.

The reason the 3rd party sites exist is that USTA does not ever disclose dNTRP. The data that he 3rd parties are grabbing is simply the match results.
 

Vox Rationis

Professional
Comparing TLS vs TR ratings there are at least 3 people I know with a difference of 0.3 - 0.4 between the two ratings sites. That is such a huge difference. Two of them are at ~3.93 on TR and 4.33 on TLS. The 3rd is at 3.97 on TR and 4.27 on TLS. Will be interesting to see if any of them get bumped down. Knowing all 3 I'd describe them as lower end 4.5s so maybe the truth is somewhere in the middle?
 
Comparing TLS vs TR ratings there are at least 3 people I know with a difference of 0.3 - 0.4 between the two ratings sites. That is such a huge difference. Two of them are at ~3.93 on TR and 4.33 on TLS. The 3rd is at 3.97 on TR and 4.27 on TLS. Will be interesting to see if any of them get bumped down. Knowing all 3 I'd describe them as lower end 4.5s so maybe the truth is somewhere in the middle?
My understanding is that TLS does not incorporate tri-level matches, but does include mixed matches.

On the other hand, TR does incorporate tri-level matches, and calculates separate ratings for mixed (thereby excluding mixed results from the regular rating).

In most of the cases that I've looked at with a large difference between the two, this has at least partially explained the difference.
 

Vox Rationis

Professional
My understanding is that TLS does not incorporate tri-level matches, but does include mixed matches.

On the other hand, TR does incorporate tri-level matches, and calculates separate ratings for mixed (thereby excluding mixed results from the regular rating).

In most of the cases that I've looked at with a large difference between the two, this has at least partially explained the difference.
In this case two of these guys only played 18+ and 40+ this year, so both sites should be working with the same data. Tri-level doesn't count in my section which TR takes into account (they don't give us ratings for our tri-level matches) so the only difference could be his mixed and combo rating. But I see nothing that indicates TLS includes mixed in his rating. If I had to guess both sites are just way off as they are for many many people.
 
In this case two of these guys only played 18+ and 40+ this year, so both sites should be working with the same data. Tri-level doesn't count in my section which TR takes into account (they don't give us ratings for our tri-level matches) so the only difference could be his mixed and combo rating. But I see nothing that indicates TLS includes mixed in his rating. If I had to guess both sites are just way off as they are for many many people.
Maybe TR is sophisticated enough to vary which leagues they include by section, because they do include tri-level for my section (though I was told that a non-advancing tri-level league I participated in last spring does not count, but the advancing tri-level league I'm currently participating in does count...but TR is counting both).

The reason I think mixed and combo mixed are being included in TLS is because they are listed under the player in their "detail rating order" page. Now, whether they only use those matches when a player doesn't have 3 regular league matches (to simulate an "M" rating), or if they always use them in their rating calc isn't specified.
 

travlerajm

Talk Tennis Guru
If found that TR and TLS have tracked fairly closely for me (except for that time when TR had an algorithm bug a couple of years ago that they resolved).
 

Vox Rationis

Professional
The reason I think mixed and combo mixed are being included in TLS is because they are listed under the player in their "detail rating order" page. Now, whether they only use those matches when a player doesn't have 3 regular league matches (to simulate an "M" rating), or if they always use them in their rating calc isn't specified.
Yeah that's what I was getting at. They list those leagues but don't specify anywhere what goes into their ratings. A little silly that we can't even tell what they're basing our estimated ratings on. No way to judge how reliable they may be.
 

time_fly

Hall of Fame
And now comes the special time of year where TR and TLS go into hibernation, or perhaps on walk-about, contemplating the great mysteries of the real bump list and seeking the errors of their ways. Expect to see updates again in late January or February.
 

Moveforwardalways

Hall of Fame
And now comes the special time of year where TR and TLS go into hibernation, or perhaps on walk-about, contemplating the great mysteries of the real bump list and seeking the errors of their ways. Expect to see updates again in late January or February.

Particularly with the increased number of bumps in some areas, TLS and TR are going to be WAY off on their ratings. If they do not adjust, it is going to have a ripple effect into the 2020 season.
 

5sets

Hall of Fame
Lol, cut the guys some slack. They are free websites to look at and nobody says you have to look at them. Use them for scouting doubles pairings or approximate strength of team individuals in comparison to one another, that’s it. They never claim to be part of usta or that their ratings are usta ratings.
 

Moveforwardalways

Hall of Fame
Lol, cut the guys some slack. They are free websites to look at and nobody says you have to look at them. Use them for scouting doubles pairings or approximate strength of team individuals in comparison to one another, that’s it. They never claim to be part of usta or that their ratings are usta ratings.

True, but the site owners are making not insignificant revenue from ads and maybe even having access to players names and cities of origin. I bet if they get a rep for being inaccurate it will hurt their bottom line.
 

5sets

Hall of Fame
True, but the site owners are making not insignificant revenue from ads and maybe even having access to players names and cities of origin. I bet if they get a rep for being inaccurate it will hurt their bottom line.
It is kind of creepy that they sent me emails as I don’t recall giving them my email address. Did they scrape that from a TW order or my usta account?
 

Moveforwardalways

Hall of Fame
It is kind of creepy that they sent me emails as I don’t recall giving them my email address. Did they scrape that from a TW order or my usta account?

They have access to more data than you might think. I have received emails from them too, and there is no way they could have my email address except for the USTA data.
 

5sets

Hall of Fame
They have access to more data than you might think. I have received emails from them too, and there is no way they could have my email address except for the USTA data.
Now I am quite sure they have our credit card numbers as well, so everyone should stop bashing them before they go on a shopping spree at your expense.
 

badmice2

Professional
Now I am quite sure they have our credit card numbers as well, so everyone should stop bashing them before they go on a shopping spree at your expense.
Funny to see all this about ad dollars. I work in the online advertising industry and can say that they don’t make astronomical revenue from ads...maybe enough to have an expensive dinner couple times a month.

To the point about scraping data, I would not be surprise if they’re scraping your browser cookie or sniffing for some sort of login info. Above all, if you do enough searches of yourself, they can probably pull your info from USTA National (I can only speculate about this, not going into details). Either way, as someone pointed out, they have way more personal data that one can expect, of which they will be sue (class action from TW board members?) due to CCPA compliance if they don’t disclose that they’re collecting and using your data without consent.
 

TennisOTM

Professional
Browsing players from my area, this year TLS did noticeably better than TennisRecord at predicting bump-ups. While they both had a fair number of misses, TLS had fewer, and their misses were less egregious.
 

Matthew ATX

Semi-Pro
TennisRecord is scraping tournament data now... kinda.
All my tournament matches are now on there. But every name associated with every match is wrong. Both partner and opponent names are people I've never heard of. The people listed in those spots are from all over the country. The scores seem to be correct for each match.
 
TennisRecord is scraping tournament data now... kinda.
All my tournament matches are now on there. But every name associated with every match is wrong. Both partner and opponent names are people I've never heard of. The people listed in those spots are from all over the country. The scores seem to be correct for each match.
Same here. It has me playing a doubles tournament with a different partner every round and each of the players on court are female, except me. Another singles tournament I played, it has me playing one of the matches as a single player playing against a mixed doubles team. All the scores seem correct for my matches too.
 

time_fly

Hall of Fame
Interesting. I wish they would start pulling the 2020 league stuff. We do Early Start in my area and I've played about 10 matches already this year. Even if the ratings for those matches aren't calculated, being able to scout for lineup purposes is much easier in TR than on Tennislink.
 

5sets

Hall of Fame
Hmmm. Yeah TR has my Men’s Open tournament matches against two females kind of like Canadian doubles? Kind of like how USTA has me listed in Men’s 60s and 70s Rankings even though I’m in my 30s.
 

5sets

Hall of Fame
In all seriousness my opponents listed in the Men’s Open Singles Tournament matches on Tennis Record are 3.0 Ladies. Bizarre.
 

time_fly

Hall of Fame
My TR page does not include any of my matches after 11/9/2019. Has anyone else noticed a lack of updates after that date?
I've been looking at the site for a few years. This is normal. They stop pulling data from USTA at the end of the league year, update the ratings, then try to fix their calculations based on the actual bumps versus their predicted ones. This seems to take some time. Once they've done that, they will start adding the new league year data.
 

Matthew ATX

Semi-Pro
I've been looking at the site for a few years. This is normal. They stop pulling data from USTA at the end of the league year, update the ratings, then try to fix their calculations based on the actual bumps versus their predicted ones. This seems to take some time. Once they've done that, they will start adding the new league year data.

Gonna take even longer for Texas and Southern this year. I'm playing 4.5's with 3.77 ratings on TR, lol.
 

time_fly

Hall of Fame
It seems like TR is no longer updating ratings based on matches when one of the players on the court is self-rated with fewer than 3 qualifying matches. For those of us playing at lower levels, this actually removes a pretty significant number of matches from consideration. I think TR’s algorithm would be more accurate if it were iterative rather than always calculating forward. I haven’t looked deeply into the UTR method but based on the fact that my rating fluctuates even without new results makes me think they calculate backwards as well as forwards.
 

Vox Rationis

Professional
It seems like TR is no longer updating ratings based on matches when one of the players on the court is self-rated with fewer than 3 qualifying matches. For those of us playing at lower levels, this actually removes a pretty significant number of matches from consideration. I think TR’s algorithm would be more accurate if it were iterative rather than always calculating forward. I haven’t looked deeply into the UTR method but based on the fact that my rating fluctuates even without new results makes me think they calculate backwards as well as forwards.
I think they get calculated later. Same thing happened last year.
 

time_fly

Hall of Fame
I think they get calculated later. Same thing happened last year.

I hope so. If done properly it's a lot more complicated than simple forward calculation given the starting ratings however. You use the matches to estimate the unrated players' ratings assuming the other players' ratings are fixed and correct. Once you have an estimate of the new player's ratings from multiple matches, you can go and plug that back in as his starting rating and recalculate the old matches -- but this will change the ratings of the other players, which means the original start estimation isn't quite right any more. You need a method that converges to ratings for all the players and matches simultaneously, and of course there's propagation of those effects across the network of other matches to consider.

Obviously I work on related data analysis problems, but to be clear, I haven't worked on or investigated player rating systems specifically. I suspect given the background of Oracle being involved in UTR that it's much more sophisticated than the USTA system. The biggest flaw in the USTA system, or at least the reverse-engineerings of it that that I have seen, is that it doesn't do enough to dampen the ratings changes of the players over short periods of time or -- often related -- eliminate outliers. Moving averages based on small numbers of samples are very susceptible to outliers, and the fact that such outliers propagate to all the other players via the network of matches is just a nasty bit of business.
 

KaiserW

Hall of Fame
Don't know about TLS but I don't think Tennis Record is that accurate. I was tracking it last season as I thought it was pretty accurate.

A couple of examples. Guy on my 4.0 team was rated 3.72 in TR and he got bumped to 4.5 and two other guys on another team were 3.73 and 3.74 and also got bumped.

So I tend to think TR in general gives lower ratings compared to actual.
 

time_fly

Hall of Fame
Don't know about TLS but I don't think Tennis Record is that accurate. I was tracking it last season as I thought it was pretty accurate.

A couple of examples. Guy on my 4.0 team was rated 3.72 in TR and he got bumped to 4.5 and two other guys on another team were 3.73 and 3.74 and also got bumped.

So I tend to think TR in general gives lower ratings compared to actual.

It's possible. Keep in mind that the USTA can decide to slide the boundaries at any time though to balance out the number of people playing at different levels. As long as none of the bins get too wide, it doesn't really matter where the lines are drawn. In our area it seemed like they slid a significant number of strong 4.0s up to 4.5. I suspect those weren't all true bumps (dNTRP calculated >= 4.0) but just a way to grow the 4.5 leagues a little bit.
 
There were also a few bugs in tennis record last year that affected some ratings. I played a match with a 3.5A and it assumed he had appealed up instead of down, so when our match dynamic ratings were calculated it gave him a 3.0 and my rating was way higher than it should be. They never corrected that. I was a 3.55 last year and wasn’t bumped despite the great Texas bump. The previous year, i was a 3.37 and was bumped but was able to appeal down.
 

time_fly

Hall of Fame
There were also a few bugs in tennis record last year that affected some ratings. I played a match with a 3.5A and it assumed he had appealed up instead of down, so when our match dynamic ratings were calculated it gave him a 3.0 and my rating was way higher than it should be. They never corrected that. I was a 3.55 last year and wasn’t bumped despite the great Texas bump. The previous year, i was a 3.37 and was bumped but was able to appeal down.

Yeah, they got the method wrong for appeals. Appealing your rating changes the level you are labeled as, but it doesn't change your dNTRP. Let's say that you have a TR NTRP of 3.30 but you now have a USTA rating of 3.5A. That means that USTA thought you were a 3.0 or a 4.0, but TR can't tell which. There are 3 possible things to do: (1) set your dNTRP to 3.25. The bump means you were above 3.5 or below 3.0, so without any other info, 3.25 will minimize the maximum possible error -- but will clearly be in error itself. (2) set your dNTRP estimate to 3.50, based on guessing from your TR 3.30 (>= 3.25) estimate that you were most likely bumped up. Depending on how accurate TR is, this could provide a lower total overall error in the estimates for "A" players. (3), do a procedure like (2) but use a weighted random guess of up versus down based on the TR estimate. Weighted randomization will help to eliminate bias in the "A" rating estimates if TR estimates are inaccurately biased themselves, but the randomization also guarantees that you will have to live with a fair amount of error (that's life). Finally, (4) is leave the rating unchanged. The bump and appeal didn't change your dNTRP, so you will neither gain nor lose error in the TR estimate by keeping the original value.

All of the above ideas have pros and cons, but arbitrarily setting everyone to the top or bottom of the level is much worse than any of the above. It distributes the error unevenly and creates a bias towards the top / bottom of the level.
 
Last edited:
Top