Can UTR be trusted to operate and accurate system?

travlerajm

Talk Tennis Guru
Sorry, I have been busy with work but this is not correct in that the rating would be switched to unreliable. That is the difference.

The system goes back in time looking at past matches. The further back in time the matches are the less reliable those results are going to be as far as the predictive nature of the rating. Thus, it will switch to unreliable if you sit for a year and don't record actual match results. They assign a %age to the reliability.

I will address your other questions tomorrow when I have time.
How do you explain Kyrgios with 100% reliable UTR rating higher than several Top 10 guys when he hasn’t played in 11 months?

Guy has never even been Top 10 himself.
 

socallefty

G.O.A.T.
Why would any system give credit to a player if his opponent improved after they played? Doesn’t make any sense to me as the match happened when the opponent was at a particular level that day - if he took daily lessons, practiced hard and improved dramatically in 6-12 months after as can happen with juniors, why would his match from before that be rated differently for himself or the other player?
 

nyta2

Legend
How do you explain Kyrgios with 100% reliable UTR rating higher than several Top 10 guys when he hasn’t played in 11 months?

Guy has never even been Top 10 himself.
guessing they use some kind of "protected utr rating" (unlike rec/juiniors that expire after X days of no play)
having a higher utr ... he did wins over guys like meddy bear (at 15.99)...
rating != ranking
 

nyta2

Legend
Why would any system give credit to a player if his opponent improved after they played? Doesn’t make any sense to me as the match happened when the opponent was at a particular level that day - if he took daily lessons, practiced hard and improved dramatically in 6-12 months after as can happen with juniors, why would his match from before that be rated differently for himself or the other player?
true, but on the flip side, how do you account for "bad days"... guessing the algo tries to smooth things out by taking some sampling (eg. like +-1mo before and after a match, or some kind of window like that)...
not saying the algo is right/perfect/etc... just saying you have to pick some kind of averaging algo to apply recursively over time?
 

socallefty

G.O.A.T.
true, but on the flip side, how do you account for "bad days"... guessing the algo tries to smooth things out by taking some sampling (eg. like +-1mo before and after a match, or some kind of window like that)...
If they had a player with an ‘unreliable’ rating, maybe they should do something. But if they have a junior with a highly reliable rating who has played a lot of matches, his performance on that day should not be “calibrated” by whether he improved months later.
 

Tennis2349

Semi-Pro
Sorry, I have been busy with work but this is not correct in that the rating would be switched to unreliable. That is the difference.

The system goes back in time looking at past matches. The further back in time the matches are the less reliable those results are going to be as far as the predictive nature of the rating. Thus, it will switch to unreliable if you sit for a year and don't record actual match results. They assign a %age to the reliability.

I will address your other questions tomorrow when I have time.
This is incorrect. It does not switch to unreliable within 6 months. Maybe after a year.

I gave a specific example where one player plays 40 matches in 6 months, does amazingly well, and has a hard earned .8 improvement.

Player b (on many graphs) over that 6 months has a bigger increase in utr without ever playing during those 6 months. But kids he played 6 months ago improved.

Secondly, how does chess ELO accomplish this? Nobody is moving up without winning. I don’t come back to playing chess 6 months later and am given 500 ELO points… Nor am I given ElO points because someone I played worked hard and improved. That would be ridiculous.

If I beat magnus Carlsen in chess I immediately get credit for the win. Nobody is saying “well we will take that victory away and the points because Magnus might have been sick”. Or “well, the points are a secret formula and you won’t even know if the match was entered. You might not go up at all. You might go up one day in the future! Keep playing!
 
Last edited:

Tennis2349

Semi-Pro
true, but on the flip side, how do you account for "bad days"... guessing the algo tries to smooth things out by taking some sampling (eg. like +-1mo before and after a match, or some kind of window like that)...
not saying the algo is right/perfect/etc... just saying you have to pick some kind of averaging algo to apply recursively over time?
Kind of silly. Chess elo doesn’t account for “bad days”. If a 6 beats a 7.5 , then the 7.5 had a bad day? If a 6 beats a 6 then it can’t be a bad day?
 

LOBALOT

Legend
This is incorrect. It does not switch to unreliable within 6 months. Maybe after a year.

I gave a specific example where one player plays 40 matches in 6 months, does amazingly well, and has a hard earned .8 improvement.

Player b (on many graphs) over that 6 months has a bigger increase in utr without ever playing during those 6 months.

Secondly, how does chess ELO accomplish this? Nobody is moving up without winning. I don’t come back to playing chess 6 months later and am given 500 ELO points… Nor am I given ElO points because someone I played worked hard and improved. That would be ridiculous.

You know you have me looking at their web site for answers instead of you doing it first. Their web site actually indicates just as I said. After a year the ratings are inactive and over that 1 year of activity the reliability goes down in decreasing reliability over that period. You gotta read and look what they are doing vs. just arguing.

The info is right here and available for you to search just as I am.

 

nyta2

Legend
If they had a player with an ‘unreliable’ rating, maybe they should do something. But if they have a junior with a highly reliable rating who has played a lot of matches, his performance on that day should not be “calibrated” by whether he improved months later.
but i'm guessing they are averaging over many data points?

if i were writing the algo, to calculate my utr (brute force!)...
* define a window to pool matches from (#days? #months? #matches)
* let's say i choose "all opponents played over the last 3 mos"...
* since there are so many data points, let's say it takes 1mo by the time it's time to recalculate me
so by the time it's time to calcmyutr... let's say the first person in than 3mos window, has played a match every day in the last month, and even got some big wins that helped boosted his utr... and has had his utr calc'd just before mine... so now when my .calcutr() is run... i'm effectively getting credit for my opponent's "wins (or losses) in the future"
 

LOBALOT

Legend
If they had a player with an ‘unreliable’ rating, maybe they should do something. But if they have a junior with a highly reliable rating who has played a lot of matches, his performance on that day should not be “calibrated” by whether he improved months later.

It doesn't their web site indicates it is a set number of matches.
 

Tennis2349

Semi-Pro
You know you have me looking at their web site for answers instead of you doing it first. Their web site actually indicates just as I said. After a year the ratings are inactive and over that 1 year of activity the reliability goes down in decreasing reliability over that period. You gotta read and look what they are doing vs. just arguing.

The info is right here and available for you to search just as I am.

I guess you haven’t read one of my posts. I said 6 months about 4 times now.

Again…. Player A does well over 6 months of competition. Goes up .8 utr. Plays 40 matches. Training. Hot. Cold. Competitions. Early weekend mornings. Several weekends spent travelling to tournaments.

Junior B sits home and does not play at all. He goes up a full utr point over the same time period.
 

travlerajm

Talk Tennis Guru
guessing they use some kind of "protected utr rating" (unlike rec/juiniors that expire after X days of no play)
having a higher utr ... he did wins over guys like meddy bear (at 15.99)...
rating != ranking
The Meddy bear match in 2022 was his last win over a main tour level player.
 

Tennis2349

Semi-Pro
The Meddy bear match in 2022 was his last win over a main tour level player.
Give people any system and many will just follow it blindly and make up excuses for something that don’t even exist. Obviously ridiculous to say Krygios has a utr that went up higher than a top ten player.
 

nyta2

Legend
The Meddy bear match in 2022 was his last win over a main tour level player.
can only guess the treat atp/wta utr differently... (else he wouldn't even have a utr after like 3mos)
appears he only played 1 match in 2023, so his rating must be mostly based on 2022...
there is a last-3mo which is blank
 

Tennis2349

Semi-Pro
can only guess the treat atp/wta utr differently... (else he wouldn't even have a utr after like 3mos)
appears he only played 1 match in 2023, so his rating must be mostly based on 2022...
there is a last-3mo which is blank
The methodology for doubles is even worse. Choose a bad partner and drop a full UTR point.

Nobody really looks at doubles UTR, so I guess those flaws are ok. We just assume singles UTR has it together.
 

nyta2

Legend
i think we're all preaching to the choir here (each other)...
no algo is gonna be perfect, but it's probably the best we have, and does a decent job approximating where you are within 1pt presuming you play a decent # of matches
in the end it's up to us adults to educate/coach kids that the path to winning tennis is not how many .01 of a utrPoint i can manipulate/accumulate per match....
it's about investing (practice/lessons!) in long term assets (weapons) that will pay dividends (match wins/utrPoints) in the long term
 

Tennis2349

Semi-Pro
Why do we accept this system when other systems like chess elo are already proven to be better?

A. Chess ElO is 100 percent free. Can see anyone’s ElO for free. Can even see all of your own analytics for free. In great detail.

B. If you have a chess victory, you know exactly what you will get. It isn’t a secret. You put in the work to get that victory. It shouldn’t matter if past opponents played poorly, so now you don’t get credit?

C. If you beat a higher ranked player you get credit. Would be ridiculous to say 2 chess players played a match, the better player lost, but he “must have been sick or had a bad day” so the result won’t count at all!

D. You aren’t given chess elo points because others improved you played months ago. Would be ridiculous to study, read, train at chess, play hundreds of games, and then notice someone who hasn’t done any of that somehow had their Elo raise magically on its own more than yours.

E. Chess Elo works amazingly well. The system can't be gamed.

F. Chess Eli you get credit for a victory within a millisecond. There is no “well thanks for playing. Now wait for a day, weeks, or maybe months , or maybe that win won’t help you at all.

G. Chess is a game of win, lose, or draw. Tennis is win or lose. The ELO system doesn’t say “well, you lost 7 pawns and a rook so that victory doesn’t count as much now You have to win by not losing any pieces “, which would totally change the game in itself.

H. Since you know what you will get, there is no chance of backend computer glitches owned by a single company that will affect your ELO. Isn’t transparency nice? Would be strange to play 20’chess games then just wait and see what happens to your rating for weeks or months. Then wonder why a computer is saying those victories aren’t meaningful, or too meaningful, and never really know what the computer expects you to actually do.

I. Chess elo also works across all ages and genders. Doesn’t matter if you are male, female, 5 or 95.

J. If you are losing a chess match, and quit, it doesn’t protect your ELO. That would be ridiculous. You don’t say “well I have a headache, so I will quit to protect my elo”. Sure you can quit, but you lose the points. Why should the victor not get any credit because you were losing and quit? Amazing how many kids have a sore leg once they start to lose. Which means you get your best victory just taken away from you (that month or every two weeks) and it won’t count.

Are we all following a number blindly because we are told to? Or are these differences existing with the UTR “modified El0” to get people to pay for it and make more money?

Yes, people say “it’s the best we have”, but why does it have to be that complex? Why not a simple, straight forward ELO system like chess has that is proven to work?
 
Last edited:

nyta2

Legend
Why do we accept this system when other systems like chess elo are already proven to be better?

A. Chess ElO is 100 percent free. Can see anyone’s ElO for free. Can even see all of your own analytics for free. In great detail.

B. If you have a chess victory, you know exactly what you will get. It isn’t a secret. You put in the work to get that victory. It shouldn’t matter if past opponents played poorly, so now you don’t get credit?

C. If you beat a higher ranked player you get credit. Would be ridiculous to say 2 chess players played a match, the better player lost, but he “must have been sick or had a bad day” so the result won’t count at all!

D. You aren’t given chess elo points because others improved you played months ago. Would be ridiculous to study, read, train at chess, play hundreds of games, and then notice someone who hasn’t done any of that somehow had their Elo raise magically on its own more than yours.

E. Chess Elo works amazingly well. The system can't be gamed.

F. Chess Eli you get credit for a victory within a millisecond. There is no “well thanks for playing. Now wait for a day, weeks, or maybe months , or maybe that win won’t help you at all.

G. Chess is a game of win, lose, or draw. Tennis is win or lose. The ELO system doesn’t say “well, you lost 7 pawns and a rook so that victory doesn’t count as much now You have to win by not losing any pieces “, which would totally change the game in itself.

H. Since you know what you will get, there is no chance of backend computer glitches owned by a single company that will affect your ELO. Isn’t transparency nice? Would be strange to play 20’chess games then just wait and see what happens to your rating for weeks or months. Then wonder why a computer is saying those victories aren’t meaningful, or too meaningful, and never really know what the computer expects you to actually do.

I. Chess elo also works across all ages and genders. Doesn’t matter if you are male, female, 5 or 95.

J. If you are losing a chess match, and quit, it doesn’t protect your ELO. That would be ridiculous. You don’t say “well I have a headache, so I will quit to protect my elo”. Sure you can quit, but you lose the points. Why should the victor not get any credit because you were losing and quit?

Are we all following a number blindly because we are told to? Or are these differences existing with the UTR “modified El0” to get people to pay for it and make more money?

Yes, people say “it’s the best we have”, but why does it have to be that complex? Why not a simple, straight forward ELO system like chess has that is proven to work?
amen to that.
always wondered why they didn't just impl the chess elo... vs. just "base the algo on chess elo"...
guessing they try to meaning from the game score (eg. close match == close in ability)
 

Tennis2349

Semi-Pro
And if you look at the graph for krygios, his last match he was a 16.18. Ok , he then went down to 15.57 because he didn’t play for 6 months.

Then not playing for another 6 months he raises up to a 15.83.

So somehow, within the hardest group to raise your UTR it goes up over a quarter point with total inactivity?

As for scholarships we would all assume a 12 utr is better than a 11.75. But as you can see, a quarter point can be given to you for no reason at all that defies logic. Can it be a half point? Or even a full point with the correct strategy? Who knows.

Ask yourself if you were playing constantly how good you would have to play to raise a quarter point once at 15 utr. Would that be possible after 10 months of inactivity?
amen to that.
always wondered why they didn't just impl the chess elo... vs. just "base the algo on chess elo"...
guessing they try to meaning from the game score (eg. close match == close in ability)
 
Last edited:

nyta2

Legend
As for scholarships we would all assume a 12 utr is better than a 11.75. But as you can see, a quarter point can be given to you for no reason at all that defies logic. Can it be a half point? Or even a full point with the correct strategy? Who knows.
no one is looking at the utr, and blindly giving scholarships... it's just a reference point, like SAT/ACT, still need to have good grades, good essay, etc... (ie. good tourney results, look good playing live... )
 

Tennis2349

Semi-Pro
no one is looking at the utr, and blindly giving scholarships... it's just a reference point, like SAT/ACT, still need to have good grades, good essay, etc... (ie. good tourney results, look good playing live... )
We are saying that is the most important component. A 12 most likely selected over an 11. Or 11.5.

Maybe. Maybe not. Not sure if coaches go to see live play. Maybe some do. Some don’t.

Overall point is that is is an important number , but also a secret that often times makes little sense.

Another discrepancy would be juniors who might be a 6 utr, but their best victory is against a 5.5.

Another junior may have the exact same utr, but have 10 victories over 6.5’s to 7. The 6.5’s he beats might not be as improving rapidly as the 5’s the other kid beats, so we say they are equal.

As for juniors I don’t even see the utility.

The USTA rankings are almost perfectly correlated with utr anyway.

Boys under 14’s in my state The top 10 might be 9 utr, the next group 8, the next 15 7 etc. you won’t have the number 1 player in the state be a 6.
 
Last edited:
this is a hilarious thread of people whining about their utr under the guise of "THE SYSTEM IS BROKEN"

guy with the green profile is constantly doing this, and has no idea how utr works or what the levels signify (e.g. in another post he said that a UTR 7 was closer to an 11 than a 3 was to a 7, LOL)

utr is based on your last 30 matches or matches played within the last year

if a player plays more matches then poor past results can drop off, leading to an increase

similarly, matches played beyond one year drop off, which can lead to a player's UTR going up without playing.

the fact is, as someone with a lot of tournament experience, who works with tournament juniors, UTR is pretty darn accurate, especially within the data set OP is concerned about (tournament-playing juniors)
 

Tennis2349

Semi-Pro
this is a hilarious thread of people whining about their utr under the guise of "THE SYSTEM IS BROKEN"

guy with the green profile is constantly doing this, and has no idea how utr works or what the levels signify (e.g. in another post he said that a UTR 7 was closer to an 11 than a 3 was to a 7, LOL)

utr is based on your last 30 matches or matches played within the last year

if a player plays more matches then poor past results can drop off, leading to an increase

similarly, matches played beyond one year drop off, which can lead to a player's UTR going up without playing.

the fact is, as someone with a lot of tournament experience, who works with tournament juniors, UTR is pretty darn accurate, especially within the data set OP is concerned about (tournament-playing juniors)
Again, this is based on the chess ElO system.

Why doesn’t chess do the same? Just give you 300 elo for not playing? Or credit YOU because a player you played before improved. Or say it doesn’t count if you beat a player that you shouldn’t. So you aren’t allowed to lol

chess Elo is extremely accurate without having all these inserted rules.
 

Tennis2349

Semi-Pro
Utr makes coaches lazy.

Real life example.

Friend of my son is in an academy. Need to be. 6utr to get into highest level of academy..

Kid truly doesn’t belong in highest level, but that the rule.

For the last 6 months parents and kid are trying anything to get it 6 utr. Playing constantly. Quitting matches while losing. Withdrawing against certain opponents. Trying to play up an age group

Then when you see how utr is calculated. It all seems ridiculous. Kid who didn’t play in 6 months went from 4.83 to 6.2.
 

nyta2

Legend
the fact is, as someone with a lot of tournament experience, who works with tournament juniors, UTR is pretty darn accurate, especially within the data set OP is concerned about (tournament-playing juniors)
yep, from my experience with junior tourneys... when seeded by utr... the seeds typically hold, where expected coinflips are usually in the <1utr differential... occasionally see an outlier of 2utr difference flip flopping, but very rare... IME
 
no one is looking at the utr, and blindly giving scholarships... it's just a reference point, like SAT/ACT, still need to have good grades, good essay, etc... (ie. good tourney results, look good playing live... )
Exactly, mostly UTR helps kids in far off places maybe get the attention of a coach instead of just sending a bunch of match video, also helps coaches have a gaitkeeper
this is a hilarious thread of people whining about their utr under the guise of "THE SYSTEM IS BROKEN"

guy with the green profile is constantly doing this, and has no idea how utr works or what the levels signify (e.g. in another post he said that a UTR 7 was closer to an 11 than a 3 was to a 7, LOL)

utr is based on your last 30 matches or matches played within the last year

if a player plays more matches then poor past results can drop off, leading to an increase

similarly, matches played beyond one year drop off, which can lead to a player's UTR going up without playing.

the fact is, as someone with a lot of tournament experience, who works with tournament juniors, UTR is pretty darn accurate, especially within the data set OP is concerned about (tournament-playing juniors)
Kudos, I have given the play more matches advice to our friend TJ, but he's played about 6 matches per year on average and still wonders why the system kind of goes sideways on his number. UTR is also not meant to reinvent mixed doubles ratings, like you said it's a nice junior sorting mechanism.
Utr makes coaches lazy.

Real life example.

Friend of my son is in an academy. Need to be. 6utr to get into highest level of academy..

Kid truly doesn’t belong in highest level, but that the rule.

For the last 6 months parents and kid are trying anything to get it 6 utr. Playing constantly. Quitting matches while losing. Withdrawing against certain opponents. Trying to play up an age group

Then when you see how utr is calculated. It all seems ridiculous. Kid who didn’t play in 6 months went from 4.83 to 6.2.
Kind of, the coach there just wants a nonnegotiable number so the parents don't bug him to death about their little superstar, if that kid gets the 6, then the coach gets the extra $$$, just a good way to do business. Imagine without UTR, there would be more parents emailing and in his ear about how their kid should be in the advanced group, look at his serve, it's advanced!
 
I'm curious about chess , it comes up alot when people are critical of tennis rating systems. Is there a short answer....are chess players obsessed with ratings so they just veer into these conversations naturally, or is the chess rating system really that good AND applicable to tennis matches where 6-0 6-0 should be counted differently than 6-4 4-6 6-4?
 

travlerajm

Talk Tennis Guru
I'm curious about chess , it comes up alot when people are critical of tennis rating systems. Is there a short answer....are chess players obsessed with ratings so they just veer into these conversations naturally, or is the chess rating system really that good AND applicable to tennis matches where 6-0 6-0 should be counted differently than 6-4 4-6 6-4?
Statistical analysis guru Nate Silver’s 538 web site used to have a sports section that kept updated ELO ratings for major sports.

It still used a dependence on the score, with sensitivity to score decaying exponentially with the score differential.

He explained that he could set the ‘k’ value that determines the score sensitivity. And that he performed regressions based on past seasons data to optimize the k.

UTR badly needs Nate.
 
Last edited:

Tennis2349

Semi-Pro
I'm curious about chess , it comes up alot when people are critical of tennis rating systems. Is there a short answer....are chess players obsessed with ratings so they just veer into these conversations naturally, or is the chess rating system really that good AND applicable to tennis matches where 6-0 6-0 should be counted differently than 6-4 4-6 6-4?
I think if we want to change the sport of tennis from a sport of sets and matches to games, then atleast it needs to be transparent. WtN tracks sets.

What percent of the algorithm gives games won preference over matches falling off, other kids you played improving etc. it says it matters, but nobody really knows.

Chess elo doesn’t count pieces left on the table and give you a higher or lower rating for victory. Atleast you see what you get for a victory. Instantly. And it is extremely accurate.

coaches love it, but they speak two faced. They tell you it means nothing, but run their entire academy based off of it.
 

travlerajm

Talk Tennis Guru
I have been a vocal critic of UTR, pointing out various obvious ‘bugs’ in the algorithm.

Two years ago, I pointed out that UTR was unable to handle mixed doubles results, which I found ironic, since the whole idea with UTR was to have a rating system that, unlike the ntrp scale, can treat both genders equally.

My doubles UTR jumped up and down that season several entire UTR points, ranging from UTR 4 on the low end, to UTR 9 on the high end, even while maintaining “100% reliability” after I had logged my minimum 5 matches that UTR says on their web site is all that is needed for a reliable rating.

I will be beginning a mixed doubles usta league this weekend, once again stressing the UTR algorithm.
 

LOBALOT

Legend
this is a hilarious thread of people whining about their utr under the guise of "THE SYSTEM IS BROKEN"

guy with the green profile is constantly doing this, and has no idea how utr works or what the levels signify (e.g. in another post he said that a UTR 7 was closer to an 11 than a 3 was to a 7, LOL)

utr is based on your last 30 matches or matches played within the last year

if a player plays more matches then poor past results can drop off, leading to an increase

similarly, matches played beyond one year drop off, which can lead to a player's UTR going up without playing.

the fact is, as someone with a lot of tournament experience, who works with tournament juniors, UTR is pretty darn accurate, especially within the data set OP is concerned about (tournament-playing juniors)

Summed up right. As I indicated at the start. It works great for juniors. Where I see it and other algorithms struggling based on players results against other players is when the scope of players do not mix geographically like adults (we tend to players in our local area banded by our NTRP) and smaller colleges that do not travel a lot out of their geographic area (again the players tend to get banded by the line they play on the team and then when they don't travel in a broad area).

Juniors tend to travel to tournaments and the mix and accuracy is much better.
 

tennis3

Hall of Fame
I have been a vocal critic of UTR, pointing out various obvious ‘bugs’ in the algorithm.
It seems like you need to look at a few things:

1) The various "datasets". Juniors vs adults, etc Things will probably be different within different datasets.
2) How accurate the distribution of UTR rankings is within that dataset at any given point in time
3) In other words, are we dealing with a "normal distribution"? Where most rankings / ratings fall in a X "confidence interval"?
4) Are the criticisms I'm reading directed at the "tails" of the distribution? Or are they being directed at the "normal distribution"?
5) Understand that by "normal" I mean players that we feel are accurately rated (or very close) and "outliers" are players we feel are "grossly" misrated.

In other words, what I'm asking is, are most ratings "close" to what we would expect them to be? In other words, do we have something like a bell curve? Or do we have significant weighting in the tails of the distribution?

I feel like with UTR and NTRP, people like to talk about the "tails". The "outliers". And there may be a good reason to do that at times. But I think it makes sense to also look at the "normal" to judge this or any system. Especially if the "normal" portion is large.

What do you think? Does UTR do a good job handling various datasets on average?
 
Last edited:

travlerajm

Talk Tennis Guru
It seems like you need to look at a few things:

1) The various "datasets". Juniors vs adults, etc Things will probably be different within different datasets.
2) How accurate the distribution of UTR rankings is within that dataset at any given point in time
3) In other words, are we dealing with a "normal distribution"? Where most rankings / ratings fall in a X "confidence interval"?
4) Are the criticisms I'm reading directed at the "tails" of the distribution? Or are they being directed at the "normal distribution"?
5) Understand that by "normal" I mean players that we feel are accurately rated (or very close) and "outliers" are players we feel are "grossly" misrated.

I feel like with UTR and NTRP, people like to talk about the "tails". The "outliers". And there may be a good reason to do that at times. But I think it makes sense to also look at the "normal" to judge this or any system.

What do you think? Does UTR do a good job handling various datasets on average?
I was referring to more glaring actual errors in the algorithm.

I detected a bug in the code for calculating doubles UTR 2 years ago only because I paid for a power level subscription, which gave me access to my performance charts.

One of these charts plotted % of games won on y-axis versus opponent strength. Opponent strength for doubles is calculated, per UTR, by adding the UTR ratings of your two opponents, then subtracting your partner’s UTR. I verified that opponent strengths on my chart were indeed calculated using this formula.

The problem was that actual match ratings were being calculated by a different formula, where they just averaged the UTR ratings of the two opponents, without partner UTR factoring into the formula. I was able to verify that my UTR match ratings lined up roughly consistent with this formula.

This error in the formula (omitting the partner strength adjustment) has minor impact on ratings for most situations, but in mixed, it causes the ratings to implode into non-possible results.

That season, I won all 6 of my 6 mixed matches, winning > 60% of the games, but my UTR rating was almost 1 UTR point lower than my opponent strength for my weakest opponent, which of course is not a logical result.
 

tennis3

Hall of Fame
I was referring to more glaring actual errors in the algorithm.


That season, I won all 6 of my 6 mixed matches, winning > 60% of the games, but my UTR rating was almost 1 UTR point lower than my opponent strength for my weakest opponent, which of course is not a logical result.
Are you saying that this "error" put you into the "tail"?

Or are you saying that this error (and other possible errors) put the entire dataset into the "tails"? Or even with this error, did it do a good job rating / ranking MIXED DOUBLE ADULTS DATASET accurately on average (in your opinion)?
 

travlerajm

Talk Tennis Guru
Are you saying that this "error" put you into the "tail"?

Or are you saying that this error (and other possible errors) put the entire dataset into the "tails"? Or even with this error, did it do a good job rating / ranking MIXED DOUBLE ADULTS DATASET accurately on average (in your opinion)?
I'm saying that this error existed at that time across the entire dataset until I pointed it out and emailed utr. It would not have been noticed if partners were similar level, but in my case, I was 5 to 6 utr units better than my partner because I was playing 8.0 mixed as a high 4.5 ntrp male playing with low 3.5 ntrp females, leading to impossible results (where 1 + 1 = -1) , so the error was obvious.
 

tennis3

Hall of Fame
I'm saying that this error existed at that time across the entire dataset until I pointed it out and emailed utr. It would not have been noticed if partners were similar level, but in my case, I was 5 to 6 utr units better than my partner because I was playing 8.0 mixed as a high 4.5 ntrp male playing with low 3.5 ntrp females, leading to impossible results (where 1 + 1 = -1) , so the error was obvious.
Anyone can make an error. Especially in a complex algorithm. I certainly don't expect perfection from this or really anything. It's probably a continuous process of refinement. And one that probably never will be perfect. Would you agree?

But even so, the question is, does it do a good job on average of rating / ranking various datasets? Especially those datasets for which there are a "sufficient number" of data points?
 
Last edited:

travlerajm

Talk Tennis Guru
Anyone can make an error. Especially in a complex algorithm. I certainly don't expect perfection from this or really anything. It's probably a continuous process of refinement. And one that probably never will be perfect. Would you agree?

But even so, the question is, does it do a good job on average of rating / ranking various datasets? Especially those datasets for which there are a "sufficient number" of data points?
I would rate UTR's performance differently for the various datasets I've paid attention to.

For rec level adult singles, I'd say it's less useful than TR, because UTR introduces arbitrary strange rules and cutoffs that bias the results.

For rec level adult mixed, UTR fails miserably, while TR handles it well.

I have a handful of racquet clients playing ITF / ATP / WTA level. UTR handles singles for ITF, challenger, and wta level matches very well (TR not applicable). An exception would be when junior players play itf matches.

For the atp pro level doubles, UTR does a relatively poor job, because it over-values opponent strength and undervalues the score differential.
 

tennis3

Hall of Fame
I would rate UTR's performance differently for the various datasets I've paid attention to.
To the OP's point, does it handle Juniors well? I always thought that it did. Obviously there will always be people that it mis-rates (at least for a time). And it can incentivize in a way that is contrary to the spirit of competition.

But on the whole, does it do a pretty good job with Juniors (looking to play in college / earn a scholarship)? My understanding is that this is the primary group UTR is intended for. The other groups are just "added revenue".
 

Tennis2349

Semi-Pro
To the OP's point, does it handle Juniors well? I always thought that it did. Obviously there will always be people that it mis-rates (at least for a time). And it can incentivize in a way that is contrary to the spirit of competition.

But on the whole, does it do a pretty good job with Juniors (looking to play in college / earn a scholarship)? My understanding is that this is the primary group UTR is intended for. The other groups are just "added revenue".
Generally speaking yes. We know if someone is an 8, they will obviously beat a 4.

But when kids are younger, and the utr is lower, 1 point is a lot.

The major thing I find strange is as I said again …

Player A over 5- 6 months plays 40 matches to improve his UTR .8

Player b sits out 6 months and improves 1 utr point! I saw one go up 1.45 in 5 months without competing. It’s a bit disheartening. My chess ELO doesn’t magically raise on its own

That sort of goes against everything utr says about raising your utr. It says you have to play often. Then we hear excuses. “Well an old match fell off”. “Well other kids improved”. Etc

It makes me wonder if for juniors, for some reason, it is weighted so they all keep going up, unless they keep competing and keep losing really badly.
 

jmnk

Hall of Fame
Generally speaking yes. We know if someone is an 8, they will obviously beat a 4.

But when kids are younger, and the utr is lower, 1 point is a lot.

The major thing I find strange is as I said again …

Player A over 5- 6 months plays 40 matches to improve his UTR .8

Player b sits out 6 months and improves 1 utr point! I saw one go up 1.45 in 5 months without competing. It’s a bit disheartening. My chess ELO doesn’t magically raise on its own

That sort of goes against everything utr says about raising your utr. It says you have to play often. Then we hear excuses. “Well an old match fell off”. “Well other kids improved”. Etc

It makes me wonder if for juniors, for some reason, it is weighted so they all keep going up, unless they keep competing and keep losing really badly.
Could you provide an actual example of a player that :
Had 100% reliable utr ranking in 6 or more utr range
Then did not play for 6 months,
And then his utr ranking increased by 1 or more points?
 

tennis3

Hall of Fame
The major thing I find strange is as I said again …

Player A over 5- 6 months plays 40 matches to improve his UTR .8

Player b sits out 6 months and improves 1 utr point! I saw one go up 1.45 in 5 months without competing. It’s a bit disheartening. My chess ELO doesn’t magically raise on its own

That sort of goes against everything utr says about raising your utr. It says you have to play often.
How do you propose it does otherwise? It will only go up without playing for a few reasons:

1) You have some old losses that age out of the calculation, so your UTR rises. I think we want this to happen. We certainly don't want 20 year old data in your UTR calculation, so it has to fall off sometime, right?

2) Your opponents UTR increases. This might not be a great reason for your UTR to rise, but how do you stop that? Can you imagine how complex the calculation would be if we not only take into account "strength of opponent", but also "date you played opponent and strength on that day"? Each person would have (a years worth?) of active UTR ratings. Maybe more.


What would your solution be to these things?
 

Tennis2349

Semi-Pro
Could you provide an actual example of a player that :
Had 100% reliable utr ranking in 6 or more utr range
Then did not play for 6 months,
And then his utr ranking increased by 1 or more points?
Yes. I don’t know how to post a photo of a graph on here.
 

Tennis2349

Semi-Pro
How do you propose it does otherwise? It will only go up without playing for a few reasons:

1) You have some old losses that age out of the calculation, so your UTR rises. I think we want this to happen. We certainly don't want 20 year old data in your UTR calculation, so it has to fall off sometime, right?

2) Your opponents UTR increases. This might not be a great reason for your UTR to rise, but how do you stop that? Can you imagine how complex the calculation would be if we not only take into account "strength of opponent", but also "date you played opponent and strength on that day"? Each person would have (a years worth?) of active UTR ratings. Maybe more.


What would your solution be to these things?
Utr is based on chess ELO. Probably the most accurate ranking system imaginable that would determine outcomes.

Your chess ElO rises because you improve and beat better players. Not because an old chess loss fell off, or because a guy I played two months ago started winning more.

Do you see the issue when a kid raises over a point in 6 months without competing? That’s a huge improvement for a junior in that amount of time. Also, how can his old utr and current both be 100 percent accurate? 1 point is a big difference

But everyone is guessing. Nobody knows. Maybe this. Maybe that. Chess you get credit for that victory immediately. There is no “well, who knows. Maybe with time!” Or “that guy you beat was too good”. Or “well, that player was 1.5 utr ahead. So it won’t count much”
 
Last edited:

Tennis2349

Semi-Pro
I would say this is what it seems like watching the group of kids my son has been playing with for a few years…

They all start at 2, move up to 3,4,5,6 etc.

If they play each other individually, my son wins easily.

But if they just stay active, and in the system, as long as they don’t completely get worse, they all rise about the same.

So when I notice one kid didn’t play at all and rose a full point, it make me wonder if the age thing is more heavily weighted than actual results.
 

travlerajm

Talk Tennis Guru
The system goes back in time looking at past matches. The further back in time the matches are the less reliable those results are going to be as far as the predictive nature of the rating. Thus, it will switch to unreliable if you sit for a year and don't record actual match results. They assign a %age to the reliability.
Verifiably false:

PS: nick’s UTR is still climbing higher each day relative to the active field. He’s on pace to enter the UTR top 10 next week.

I’ve seen similar weird trend with my own UTR during periods of inactivity.
 
Last edited:

travlerajm

Talk Tennis Guru
The elephant in the room is that UTR didn’t think through the consequences and pitfalls of adjusting everyone’s UTR based on how past opponents perform after the match occurred.

It destabilizes the whole system, so that half the UTR’s are based on fake assumed ratings (with the assumptions right in some cases but fallacious in others) rather than real ratings. It gives only the illusion of accuracy.
 

Tennis2349

Semi-Pro
The elephant in the room is that UTR didn’t think through the consequences and pitfalls of adjusting everyone’s UTR based on how past opponents perform after the match occurred.

It destabilizes the whole system, so that half the UTR’s are based on fake assumed ratings (with the assumptions right in some cases but fallacious in others) rather than real ratings. It gives only the illusion of accuracy.
exactly …

This is how I think they handle it…for juniors it will go up basically no matter what.

Player in question:

October stops competing as a 4.53

March competes again as a 6.1!

(Magically went up)

Loses to a 4 utr. Well, we already know that over 2 utr point matches aren’t even counted.

So he stays at. 6.1.

Loses to a 4.5 next.

Stays a 6.1! Utr does not weigh large gaps heavily.

Enters a level 6 tournament and beat a 16 yr old 6.

Utr goes up to 6.2.

The 2 lower ranked players gained nothing from beating him.
 
Top