Data, Opinions and Schmke's blog

[Moon Shooter]

Edit: this thread is for anyone that wants to talk about or analyze data to see if it supports various opinions and also to think about what sort of data might prove or disprove certain opinions.


In this post I talked about how I would like to see certain data. As it turns out Schmke has collected quite a bit of data along the lines I was looking for.

This shows the year end ratings by his estimated dynamic rating every tenth of a point:

Interesting USTA League Stats - Histogram of NTRP ratings down to the tenth

Next in my series of posts on interesting stats from the data I collect for the Estimated Dynamic NTRP Rating reports I generate, I took...

computerratings.blogspot.com

So I am not sure if he rounds up or down and I don't have the actual numbers but in any case just by eyeballing it it seems that the lower 2 fifths of 4.5 players seem to have more players than the upper 3 fifths of 4.5 players.

This would lead me to think the median 4.5 player should be about a 4.17 or so. Maybe the average 4.5 player rating would be about a 4.20.

Now these numbers are the dynamic ratings of people at the end of the year. So there may be some players that are outside these 5/10ths that are also 4.5 level players. But again since it is a level at the upper 10% of players you would think those 4.5 players that have a dynamic rating below 4.01 would greatly outnumber those with a dynamic rating above 4.50. At least all things being equal.

What are some things that might not make all things equal?

1) Appeals. Now if appeals were solely based on a persons honest judgment of their level and not to manipulate ratings so they could win we would likely expect about the same number of appeal ups as appeal downs. If the margins to appeal up are the same as the margins to appeal down then in the 4.5 level we would expect appeals process to lower both the median and the likely the mean rating. That is because there are just many more players that are say 3.93-4.01 than there are players that are 4.51-4.59. Also when we consider that most people tend to over rate their own abilities (such as most people think they are above average intelligence) we might even think that if the appeals process was really just for people honestly trying to correct the ratings it would cause the median and average to drop even lower. So maybe the median would be 4.15 and the average 4.19 or something.

But in this blog he found this regarding 4.5 players:

"First, I took a look at the average dynamic rating I calculated for all players that ended the 2013 year rated a 4.5 and found it to be 4.23."

More interesting USTA League stats - Comparing the 40 & over and 18 & over divisions

In 2013, the USTA introduced new age divisions in the Adult leagues, a new 40 & over league to go along with the existing 18 & over league, ...

computerratings.blogspot.com

Now that is surprising. How can the average be so much higher than what we would think based on the relative number of people in this level? I mean you would think the average player is a 4.5 player if the average rating in that level happens to be 2.5 hundredths of a point away from the middle of the level.

Here are some thoughts:

A) Computer rated players shouldn't be causing this. Because computer rated players would seem to be as likely to go down as up when we are at this level and given the age of the average 4.5 level player. But I could be wrong.

B) People are appealing down much more than they appeal up. But as I explain in 1 above if the appeal process is being used honestly it should result in lowering the average rating of a player because there are simply many more players below this level then above it and human nature is to overestimate our ability not underestimate it. The only reason the appeals process would work to raise the average rating of a 4.5 player would be if they were appealing down in order to manipulate their rating. If someone rated down due to an injury and that injury in fact meant they were no longer a 5.0 player they would not have a year end dynamic rating pushing the 4.5 rating higher. Remember the first chart was the number of players at each .1 rating at the end of the year.

C) self rates are out of level and they are done to manipulate the rating. Again if people were just making honest mistakes in self rating you would think far more people below 4.5 would think they are 4.5 rather than 5.0s mistakenly thinking they are 4.5. Again this is because 1)there are far more people below 4.5 to miscalculate and 2) human nature tends to lead us to over estimate our ability rather than under estimate our ability. So if the self rating is pulling the anticipated average rating higher then the only cause I can think of is rating manipulation.

If I am understanding the data correctly then it seems the appeals and or self rating system are significantly distorting USTA leagues. I would be interested in any other thoughts on this data and where my analysis may be going wrong.

 

[J011yroger]

Subscribed.

J

 

[Moon Shooter]

Here is another blog with some interesting data:

Do stronger players in USTA League play singles or doubles? Interesting Tennis League Stats

In my work generating reports but also in questions I get in e-mail in response to my NTRP Ratings FAQ , a question I often get asked is if...

computerratings.blogspot.com

So this seems to show that singles players typically are higher rated than doubles players. It also seems to show that women players are lower in their level than male players. This is consistent with other data that showed women players are less likely to underestimate their rating then men when it comes to self rating.

Do USTA League self-rated players self-rate correctly? Do men sandbag and women over rate?

2016 USTA League Nationals are going on in full force, and unfortunately that means a few cases of questionable self-rated players comes up...

computerratings.blogspot.com

"However, I've told lots of people that singles can offer more opportunity to improve your rating as stronger players typically play singles. This is the conventional wisdom, that the best players play singles, so beating them will result in a better match rating than beating lower rated players on a doubles court."

Now what I wonder is whether you will improve your rating by playing better players. I think it is true to some extent. Because if a player is much higher rated you likely will not lose any rating points even if it is a worse case scenario where you lose 6-0 6-0. But you will gain rating points if you even take one game. So if someone only played those matches their rating could only improve. But I wonder if outside of those extreme cases it really matters. So in matches where even the weaker player could lose rating points is it still better to play stronger players? In chess it really didn't help. But maybe in tennis players don't really care if they win 6-0 6-0 versus 6-2 6-3 so you can gain rating points by simply playing up. What are your thoughts?

 

[leech]

Now what I wonder is whether you will improve your rating by playing better players. I think it is true to some extent. Because if a player is much higher rated you likely will not lose any rating points even if it is a worse case scenario where you lose 6-0 6-0. But you will gain rating points if you even take one game. So if someone only played those matches their rating could only improve. But I wonder if outside of those extreme cases it really matters. So in matches where even the weaker player could lose rating points is it still better to play stronger players? In chess it really didn't help. But maybe in tennis players don't really care if they win 6-0 6-0 versus 6-2 6-3 so you can gain rating points by simply playing up. What are your thoughts?

According to UTR, my four highest-rated matches were losses to high rated opponents, and my six lowest-rated matches were wins vs low rated opponents. If the USTA algorithm is remotely similar to UTR's, I would assume that an easy way for a 3.5 player, for example, to increase his/her rating is by playing up vs. strong 4.0 players.

 

[OnTheLine]

According to UTR, my four highest-rated matches were losses to high rated opponents, and my six lowest-rated matches were wins vs low rated opponents. If the USTA algorithm is remotely similar to UTR's, I would assume that an easy way for a 3.5 player, for example, to increase his/her rating is by playing up vs. strong 4.0 players.

Yup. This is why NTRP ratings are very very sticky unless you are playing "up" and at the same time not giving away games while at level.

 

[ChaelAZ]

Not exactly what it is your are really trying to accomplish amigo, but do a search on that blog about appeals and such. You are making Mt. Everest out of an ant mound there. The amount of shenanigans across the USTA leagues is very little, but becomes apparent in a few teams at Nationals.

 

[Creighton]

Yup. This is why NTRP ratings are very very sticky unless you are playing "up" and at the same time not giving away games while at level.

This is why the system really does need some tweaking. It allows people to spend years sandbagging at a level and punishes people who do the right thing and play up.

 

[ShaunS]

Now what I wonder is whether you will improve your rating by playing better players. I think it is true to some extent. Because if a player is much higher rated you likely will not lose any rating points even if it is a worse case scenario where you lose 6-0 6-0. But you will gain rating points if you even take one game. So if someone only played those matches their rating could only improve. But I wonder if outside of those extreme cases it really matters. So in matches where even the weaker player could lose rating points is it still better to play stronger players? In chess it really didn't help. But maybe in tennis players don't really care if they win 6-0 6-0 versus 6-2 6-3 so you can gain rating points by simply playing up. What are your thoughts?

While first acknowledging that I don't have the hard data to back this up, I can say there's a reasonable logic to some mismatch theory. It doesn't even have to be singles vs doubles, but it's helpful if you're a higher rated player in the range seeking a similar opponent.

If I'm playing against an opponent who is close to my level, I am going to be required to play my best tennis to win. More focused, better shots, the works. The result is that I *will* be playing better tennis, and again assuming the rating system is effective that will be reflected in a higher match rating. On the other hand, if I'm playing someone who is well below my level, and it really isn't going to be a competitive match that same drive won't be there. So subconsciously, I'm not engaged to play my best tennis. There's also the reality that I've got no interest in obliterating some guy 6-0, 6-0. That isn't fun for me, and it isn't fun for them. That isn't to say that I'm going to "toy" with them, but I'm not going to ace a guy all match long just because I can. I'd rather hit a few balls, so I'll put a tough serve within reach.

In my last 150 matches, I've won 72.5% of them. I have zero double bagels, and one 6-1, 6-0 finish. It was at districts (opponents were both under level for their team), and probably the only reason that was such a blowout was in case we needed the tiebreaker points. I just can't have fun blowing somebody off the court, and the rating system has no way to factor that in.

I can imagine there's a similar logic for the lower rated player. They want to sneak a game or two in to feel like they've executed on some things, and that'll drive them to try hard.

I understand how this can taint the larger dataset, but it's simple human nature not some grand point shaving scheme. As for addressing it, I have no solutions.

 

[Moon Shooter]

Not exactly what it is your are really trying to accomplish amigo, but do a search on that blog about appeals and such. You are making Mt. Everest out of an ant mound there.

People can disagree on whether a mile is a long distance or a short distance. But before you try to resolve that argument you might want to quantify/clarify that a mile is 63,360 inches. I am using data to try to quantify/clarify how much appeals and self rates distort the USTA levels. Whether you think the self rating and appeal processes are justified despite this distortion is up to you.

I am just trying to create a thread where people don't just make claims with no data. I don't always agree with Schmke's opinions about data but at least he offers data in support of his claims.

"In 2019, there were over 41K self-rates that got a 2019 year-end rating. Of these, 63% stayed the same level, 23% were bumped up, and 14% bumped down."
https://computerratings.blogspot.com/2020/09/what-will-effects-of-no-2020-year-end.html

From the same blog we get more data:
"In 2019, about 85% of established players stayed the same level, 5% were bumped down, and 10% bumped up."

Schmke is discussing a different issue but the data doesn't care. The data is still informative to this issue. Now if we assume that some of the self rates just naturally improved or got worse and that explained the bump or dump. But that rate should not be so different than the rate for established rated players. Yet we find that the number of self rates going up or down is more than double the number of established players. So even if we deduct the 15% - the amount we can say will typically happen just from a year of improvement (even though self rates probably join mid year more than other established players) that still leaves 22% of self rates rating themselves incorrectly. For 2019 that's about 9,000 players playing out of level due to self rating.

There were about 126,000 players you can say it is not "significant" when I would say it is "significant." But at least we have an idea of the actual quantity of the distortion.

I also predict that there will be a higher percentage of self rates in 2021 than there was in 2019 thus self rating will be an even larger distortion.

The amount of shenanigans across the USTA leagues is very little, but becomes apparent in a few teams at Nationals.

Ok that is your subjective view. But if you want to tell me it is only a few teams at nationals I would like to see the data. I think the problem is bigger, especially for men's teams. But if you could show me data that only 25% of the men's teams that competed in nationals had a self rate or appeal rate player as one of their top five players helping them get there then ok. I would find that pretty persuasive that this issue only involves "a few teams at nationals." But I suspect a higher percentage of teams even competing at state and regionals had at least one self rate or appeal rated player that performed in the top 5 helping them advance. I admit it is just a suspicion based on an just perusing some of tennis record, and I would happy to have the data correct me.

 

[Creighton]

People can disagree on whether a mile is a long distance or a short distance. But before you try to resolve that argument you might want to quantify/clarify that a mile is 63,360 inches. I am using data to try to quantify/clarify how much appeals and self rates distort the USTA levels. Whether you think the self rating and appeal processes are justified despite this distortion is up to you.

I am just trying to create a thread where people don't just make claims with no data. I don't always agree with Schmke's opinions about data but at least he offers data in support of his claims.

"In 2019, there were over 41K self-rates that got a 2019 year-end rating. Of these, 63% stayed the same level, 23% were bumped up, and 14% bumped down."
https://computerratings.blogspot.com/2020/09/what-will-effects-of-no-2020-year-end.html

From the same blog we get more data:
"In 2019, about 85% of established players stayed the same level, 5% were bumped down, and 10% bumped up."

Schmke is discussing a different issue but the data doesn't care. The data is still informative to this issue. Now if we assume that some of the self rates just naturally improved or got worse and that explained the bump or dump. But that rate should not be so different than the rate for established rated players. Yet we find that the number of self rates going up or down is more than double the number of established players. So even if we deduct the 15% - the amount we can say will typically happen just from a year of improvement (even though self rates probably join mid year more than other established players) that still leaves 22% of self rates rating themselves incorrectly. For 2019 that's about 9,000 players playing out of level due to self rating.

There were about 126,000 players you can say it is not "significant" when I would say it is "significant." But at least we have an idea of the actual quantity of the distortion.

I also predict that there will be a higher percentage of self rates in 2021 than there was in 2019 thus self rating will be an even larger distortion.



Ok that is your subjective view. But if you want to tell me it is only a few teams at nationals I would like to see the data. I think the problem is bigger, especially for men's teams. But if you could show me data that only 25% of the men's teams that competed in nationals had a self rate or appeal rate player as one of their top five players helping them get there then ok. I would find that pretty persuasive that this issue only involves "a few teams at nationals." But I suspect a higher percentage of teams even competing at state and regionals had at least one self rate or appeal rated player that performed in the top 5 helping them advance. I admit it is just a suspicion based on an just perusing some of tennis record, and I would happy to have the data correct me.

Congratulations, you've decoded the mystery that you have to have players out of level to win.

 

[jmnk]

While first acknowledging that I don't have the hard data to back this up, I can say there's a reasonable logic to some mismatch theory. It doesn't even have to be singles vs doubles, but it's helpful if you're a higher rated player in the range seeking a similar opponent.

If I'm playing against an opponent who is close to my level, I am going to be required to play my best tennis to win. More focused, better shots, the works. The result is that I *will* be playing better tennis, and again assuming the rating system is effective that will be reflected in a higher match rating. On the other hand, if I'm playing someone who is well below my level, and it really isn't going to be a competitive match that same drive won't be there. So subconsciously, I'm not engaged to play my best tennis. There's also the reality that I've got no interest in obliterating some guy 6-0, 6-0. That isn't fun for me, and it isn't fun for them. That isn't to say that I'm going to "toy" with them, but I'm not going to ace a guy all match long just because I can. I'd rather hit a few balls, so I'll put a tough serve within reach.

In my last 150 matches, I've won 72.5% of them. I have zero double bagels, and one 6-1, 6-0 finish. It was at districts (opponents were both under level for their team), and probably the only reason that was such a blowout was in case we needed the tiebreaker points. I just can't have fun blowing somebody off the court, and the rating system has no way to factor that in.

I can imagine there's a similar logic for the lower rated player. They want to sneak a game or two in to feel like they've executed on some things, and that'll drive them to try hard.

I understand how this can taint the larger dataset, but it's simple human nature not some grand point shaving scheme. As for addressing it, I have no solutions.

this is 100% correct take on practicality of a tennis match. The problem with systems that USTA or UTR use is that the ranking is _not_ based on what the goal of the match is. The goal is to win. What score you win by is a secondary and not-so-important thing - but it is the most important thing for ranking purposes. I would not be surprised if statistics showed that there are _more_ lopsided scores in matches where two opponents are _very_ equal. Because in those matches you play each point like it matters, and even if the opponent has a bad day you never let up. When you play someone below your level there's really no urgency ever - you can be down 0:3 or 1:4, or even a set - but you just know you are going to win. So maybe you do not run for that lucky net cord on break point. Maybe you try a cute drop shot volley and miss.

USTA and UTR systems essentially reward a so-called 'moral victory' - a loss, but not _that bad_ of a loss. Which is arguably the worst thing you can do in athletic competition. When you think about it those systems treat each game as a measurable outcome, as if it mattered on its own - which is obviously quite the contrary to the beauty of tennis scoring system.

 

[schmke]

this is 100% correct take on practicality of a tennis match. The problem with systems that USTA or UTR use is that the ranking is _not_ based on what the goal of the match is. The goal is to win. What score you win by is a secondary and not-so-important thing - but it is the most important thing for ranking purposes. I would not be surprised if statistics showed that there are _more_ lopsided scores in matches where two opponents are _very_ equal. Because in those matches you play each point like it matters, and even if the opponent has a bad day you never let up. When you play someone below your level there's really no urgency ever - you can be down 0:3 or 1:4, or even a set - but you just know you are going to win. So maybe you do not run for that lucky net cord on break point. Maybe you try a cute drop shot volley and miss.

USTA and UTR systems essentially reward a so-called 'moral victory' - a loss, but not _that bad_ of a loss. Which is arguably the worst thing you can do in athletic competition. When you think about it those systems treat each game as a measurable outcome, as if it mattered on its own - which is obviously quite the contrary to the beauty of tennis scoring system.

Ratings systems that look strictly at who wins/loses generally perform worse in predicting games/matches than those that consider the score.

 

[onehandbh]

Would it be harder for a person to get bumped up if they only played doubles?

 

[Moon Shooter]

I think there are some good points being made about the goals of tennis players and how games won may not always translate well. ITF looks at sets. But ultimately I think the best rating systems will want to at least consider games because it is data and rating systems need data. Whether they could weight games and sets and matches won differently is another question. But I doubt complete ignoring match score (or even set score like itf) will increase accuracy.

onehandedbh I think it may depend on your partner. Playing with a partner with a very low dynamic rating may be similar to playing up. But I also think that if someone you play with or against does not have an established rating your rating won’t change. And that is more likely to happen with 4 players then with 2. Schmke did a blog on this:

Do stronger players in USTA League play singles or doubles? Interesting Tennis League Stats

In my work generating reports but also in questions I get in e-mail in response to my NTRP Ratings FAQ , a question I often get asked is if...

computerratings.blogspot.com

Since singles players tend to have a higher dynamic rating it may be that playing singles will also be similar to playing up - again just to a lesser degree. I’m not sure if we have a better way to test whether playing singles will improve your rating with data.

 

[jmnk]

Ratings systems that look strictly at who wins/loses generally perform worse in predicting games/matches than those that consider the score.

that's interesting, I did not know that. I know college tennis ranking uses wins/losses only. So does FIFA now, in the past they did consider the score of the match but they do not anymore. What other system out there uses wins/losses only?

I'm also not necessarily advocating for going by win/loss _only_. But it needs to be part of the equation.

 

[schmke]

that's interesting, I did not know that. I know college tennis ranking uses wins/losses only. So does FIFA now, in the past they did consider the score of the match but they do not anymore. What other system out there uses wins/losses only?

I'm also not necessarily advocating for going by win/loss _only_. But it needs to be part of the equation.

Back when the BCS used a computer component, half of the systems used strictly win/loss and the others used scores to varying degrees, and if you looked at the predictive qualities, those that used the score fared better.

But, when talking about rating systems, you have to state what your goal is. Is it to be:

• Best at predicting future games
• Best "fit" for games played (or put another way maximize how well the current ratings predict prior games)
• Reflect who has accomplished the most to date (who has the best "body of work")

A given rating system won't necessarily be the best at all three.

In the USTA's case, they aren't necessarily trying to do any of the above, instead are aiming to have players of the same level be "compatible". Of course, this is the goal of the NTRP level and the dynamic rating is a different thing.

 

[Moon Shooter]

But, when talking about rating systems, you have to state what your goal is. Is it to be:

Best at predicting future games

Best "fit" for games played (or put another way maximize how well the current ratings predict prior games)

Reflect who has accomplished the most to date (who has the best "body of work")

A given rating system won't necessarily be the best at all three.

In the USTA's case, they aren't necessarily trying to do any of the above, instead are aiming to have players of the same level be "compatible". Of course, this is the goal of the NTRP level and the dynamic rating is a different thing

I am not sure what you have in mind with the second. It would seem all rating systems are based on prior games so to that extent they are simply a number that you spit out based on those results. But it may be true that rating systems that weigh more recent games more heavily would vary in their ability to rate past games. And UTR makes no attempt at all to give any idea of how good someone was over 12 months ago. Is that that distinction you are drawing?

If you are trying to find players that would be competitive then the first goal would be pretty much be the only concern.

As far as "body of work" I guess that would be more focused on matches won and lost and strength of opponent or prestige of the event as opposed to the game by game rating.

I do suspect the levels are somewhat sticky. That is once you are in a level you may be unlikely to get moved up or down. Or put another way the same player could be a low level 4.0 and never get bumped down or they could be a high level 3.5 and not get bumped up. It seems that one way to test this is to see if people that play up tend to increase their rating more than people that play at their same level.


Now it could be that people that play up may be doing that just because the team needs some extra players or they may be doing that because the player thinks they are improving and therefore belongs at the higher level. But Leech made an interesting observation about his UTR. UTR tells you which matches boosted your ratings the most. And for him it was the losses to higher rated players. Schmke would it be fair to say you had an underlying assumption that if you play opponents with higher dynamic ratings you are more likely to increase your NTRP rating? I say this because you say:

"....a question I often get asked is if a player should play singles or doubles to improve their rating the most. There is no clear answer, it all depends on the ratings of the players in the match and the score, and some players are simply better singles players than doubles or vice versa, so there is no golden rule.

However, I've told lots of people that singles can offer more opportunity to improve your rating as stronger players typically play singles. This is the conventional wisdom, that the best players play singles, so beating them will result in a better match rating than beating lower rated players on a doubles court. There are exceptions to this of course as the rating of your partner matters too, but I wanted to test this conventional wisdom to see if it was indeed true."

Do stronger players in USTA League play singles or doubles? Interesting Tennis League Stats

In my work generating reports but also in questions I get in e-mail in response to my NTRP Ratings FAQ , a question I often get asked is if...

computerratings.blogspot.com

You then show that singles players tend to have a higher rating as though this would mean playing singles courts is a better way to improve your rating (I assume here "improve" your rating means raise your rating as opposed to just making it more accurate) than doubles play.
You also say

"In the charts below, I show the average for each NTRP level showing 1S, 2S, 1D, 2D, then 3D in the groupings. I've also normalized the ratings to be relative to the bottom of each level so that the relative strength within the levels can easily be compared."

Are you saying you took the average dynamic rating of the players at each level? So for example, on men's 3.5 third doubles court the average dynamic rating is 3.15?

 

[Cashman]

In the USTA's case, they aren't necessarily trying to do any of the above, instead are aiming to have players of the same level be "compatible". Of course, this is the goal of the NTRP level and the dynamic rating is a different thing.

I am no expert in tennis ratings, but I would have thought that it would be very difficult to produce an accurate computerised rating system given:

1. the relatively small number of recorded matches that recreational players participate in
2. the lack of crossover between most playing groups

A modest goal of bucketing people into fairly wide levels with heuristics that allow as many matches as possible to be relatively competitive seems like the most sensible way to treat ratings. They won't get it right all the time but the hope is that they would get it right more often than not, and the rest is hopefully a small enough number that it can be dealt with via appeals, self-rates and the like.

I woudn't think that any dynamic rating system can be meaningfully interpreted (in terms of its movements up and down on a continuous basis) but perhaps that is just because I don't know the system well enough.

 

[Moon Shooter]

Cashman, both considerations are very important to a good national/world rating system. The first is a problem, and schmke just did a blog on the second issue here:

Is Intermountain going to bear the brunt of the USTA's 2021 bump ups?

At year-end when the USTA does their year-end ratings and players are bumped up or down, not all sections are treated equally. Some section...

computerratings.blogspot.com

As for the first issue (and indirectly the second issue) is a problem of the shortage of data. They expand the data by including the scores of matches. But there is also the question of how many matches should count toward your rating. USTA seems willing to sacrifice rating accuracy in order to prevent sandbagging. They severely limit the number of matches that they count toward ratings. They don't allow non-usta matches to effect their rating. And even with USTA matches they never allow mixed gender matches to effect a persons main rating and I believe they do not even rate some of their own usta tournaments. No mixed doubles are rated unless you only play mixed doubles but that doesn't effect your main rating. And even if a tournament is rated I don't think the match will be rated if you play someone that has a different gender. USTA is apparently afraid that adult rec players will throw games in these matches in order to tank their rating so they have a better shot at winning state or nationals. So they are intentionally limiting the data.

UTR rates all the USTA matches - even the ones USTA does not count - as well as just about any match. They will even allow people to post their own scores for completely private matches as long as both players confirm the score for an "unverified rating." This allows UTR to have a larger pool of data and it is much more accurate in areas like mine. I wish USTA would do this unverified ratings as well. But I am sure they would find that for the vast majority of players (that are not throwing games) the unverified rating would be much more accurate than the verified rating.

However UTR limits the data in ways USTA does not. 1) UTR completely separates doubles and singles and allows no overlap. So an adult that plays 10 matches per year might have only 7 count toward doubles and their singles rating will be based only on 3 matches. And that assumes all three people they played in singles had enough matches to have a rating in UTR. 2)This separation of doubles and singles may be ok generally but this problem is compounded because UTR only counts matches for the past 12 months! This might make some sense for juniors that are improving every year but for adults that play only a few rated matches a year it means the amount of data in the pool is very small. It is unclear to me why they do not just weight the older games less as more recent games come in. If no recent games come in then the old rating is better than no rating at least for 4 years or so. Greatly increasing the data pool in this way would greatly increase the accuracy of UTR for adult rec tennis and it likely wouldn't hurt Junior player ratings as long as the more recent matches are weighted more.

I think the modest goal you outline can easily be achieved. Although USTA has even given up on that now saying the goal is not to have matches be "competitive" even at the same level but only that the players be "compatible" which apparently means a 6-0 6-0 result may be typical for some matchups. I believe a good rating system can not only achieve your modest goal but can do much more and indeed even the USTA and UTR as flawed as they are already do much more. UTR can give you good information about a players strength. USTA dynamic ratings do as well its just that they do not publish that information. There are limitations built into tennis itself, but if either system really decided to make a rating systems as accurate as possible for adult rec tennis these ratings could be very good. The fact is neither UTR nor USTA really cares much about adult rec tennis. UTR is focused on being a college recruiting tool and USTA financials show adult rec tennis is a small fraction of their concern.

Does Canada have a rating system?

 

[Cashman]

There are limitations built into tennis itself, but if either system really decided to make a rating systems as accurate as possible for adult rec tennis these ratings could be very good.

I am pretty skeptical of this. With most recreational players playing only a handful of recorded matches a year, and very few playing anyone outside their local league or geographical area, the data set just doesn’t seem adequate to create a unified system that has much accuracy

You will obviously have pockets of very dedicated players who create a really dense network of matches, and this can generate good data, but this becomes more akin to a local ranking system than a broad unified rating system

I am honestly surprised that the current rating systems work as well as they do

 

[FuzzyYellowBalls]

I am pretty skeptical of this. With most recreational players playing only a handful of recorded matches a year, and very few playing anyone outside their local league or geographical area, the data set just doesn’t seem adequate to create a unified system that has much accuracy

You will obviously have pockets of very dedicated players who create a really dense network of matches, and this can generate good data, but this becomes more akin to a local ranking system than a broad unified rating system

I am honestly surprised that the current rating systems work as well as they do

That's a really good observation, better written than some of my word vomits about different areas being slightly different in ratings and my firm belief that 1 nationals event per year doesn't trickle down to every local area and set things straight. One thing that really helps the USTA, what levels are really relevant to the "very few playing anyone outside their local league". I would suggest those players are overwhelmingly 3.0-3.5, a decent chunk but not quite as large at 4.0, but 4.5 and up are rarely isolated or ignorant of what level they are.

So, the system works, but some credit goes simply to casual rec players mostly being 3.0-3.5, hard to be wrong when that large group is 2 categories, maybe 3.

 

[Moon Shooter]

I am pretty skeptical of this. With most recreational players playing only a handful of recorded matches a year, and very few playing anyone outside their local league or geographical area, the data set just doesn’t seem adequate to create a unified system that has much accuracy

You will obviously have pockets of very dedicated players who create a really dense network of matches, and this can generate good data, but this becomes more akin to a local ranking system than a broad unified rating system

I am honestly surprised that the current rating systems work as well as they do

The lack of data is a self inflicted wound because USTA refuses to record/rate the vast majority of matches.

Even the pockets of players doesn't really work well because you can't even play rated matches against the people in your own town unless you create two separate teams at the same level and have them compete in the same league. Mixed doubles games are excluded. So you need about 26 people of the same level and gender willing to play usta tennis in order to have 2 teams. For most towns you are lucky if you can get one team. USTA allows the rating system to very imprecise because they are worried people will sandbag for nationals if they rate more games. Tournaments would work but tournaments depend on a good rating system to gain popularity. So the problems snowball.

As far as issues with local pockets, college students would be a great way to have people of various levels playing in different locations! But if the draconian guidelines posted elsewhere are imposed (anyone under 25 that played even d3 tennis must self rate at 5.0 minimum!) It is no surprise that the vast majority of college players and fresh outs do not play usta tennis. So again another self inflicted wound to the rating system due to their concern over people sandbagging nationals. I'm in a local ladder with several recent d3 grads. I hope they are not all 5.0 players. If they are, we have way too much court time reserved.

That said people can play in more than one state. I am in central Illinois and there is a 3.5 team from Quincy that has quite a few players that also play in Iowa. So it is not entirely dependent on post season.

 

[Cashman]

Never mind

 

[Creighton]

The lack of data is a self inflicted wound because USTA refuses to record/rate the vast majority of matches.

What you call self-inflicted wound most of us would call protecting integrity of the data.

But if the draconian guidelines posted elsewhere are imposed (anyone under 25 that played even d3 tennis must self rate at 5.0 minimum!)

What you call draconian, most experienced tennis players would call reasonable.

So again another self inflicted wound to the rating system due to their concern over people sandbagging nationals.

Didn't this literally just happen in your area?

I'm in a local ladder with several recent d3 grads. I hope they are not all 5.0 players. If they are, we have way too much court time reserved.

This is the problem. They're going to be so far out of your level there won't be any valuable data to come from this beatdown.

 

[Moon Shooter]

Cashman
Schmke offered some data about how well the ntrp predicts outcomes and it works pretty well. I have never played a USTA single gender match - which is the only match that goes to your main USTA rating. I have played 5 usta mixed matches and about 10 non usta matches. If I end up playing in a single gender league I will probably play about 6-9 matches that USTA will rate through that league. I hope to play 3 to 5 times that many matches next year.

Ratings can be a reason to play matches. I think tennis has not had a good rating system so players don't realize that. If we had a good rating system people would play more matches in tournaments or just for fun. Without a rating system there is no overarching reason to play a match as opposed to just play a few games or a set. In any case the ratings already work pretty well even with a fraction of the data. We both agree it is likely data starved, so adding data would improve it.

Creighton
Sorry but no the d3 players wtih a 4.XX or 5.XX utr are not minimum 5.0 ntrp players. And it is not reasonable to insist that is the case, unless you want to prevent young adults, that like tennis, from joining USTA.

 

[Creighton]

Creighton
Sorry but no the d3 players wtih a 4.XX or 5.XX utr are not minimum 5.0 ntrp players. And it is not reasonable to insist that is the case, unless you want to prevent young adults, that like tennis, from joining USTA.

Any D3 player who is legitimately a UTR 4 would just self rate incorrectly and no one would complain.

 

[Moveforwardalways]

Not exactly what it is your are really trying to accomplish amigo, but do a search on that blog about appeals and such. You are making Mt. Everest out of an ant mound there. The amount of shenanigans across the USTA leagues is very little, but becomes apparent in a few teams at Nationals.

Totally agree. Self rated and appeal rated players are not skewing the ratings much at all, and they are not keeping well deserving players from getting bumped up. If you want to get bumped up, then win more matches at your current level by a wider margin. That’s how almost everyone who gets bumped does so, it’s really that simple.

 

[Moon Shooter]

This is the problem. They're going to be so far out of your level there won't be any valuable data to come from this beatdown.

I just played the match and lost 6-1 7-6. The first set was not as much of a blow out as it appeared (I got to 40 in about 5 of the 6 wins he won) but the second set was not as close as it appeared either as I felt like I was on the defense most of the time. If I had to guess I would say typical set would be 6-3.

It was definitely a learning experience for me to play a match with someone that had been coached and trained in college for 4 years. He quickly moved up and punished weak short balls much more severely and reliably than anyone I had previously played a match against. He also hit very deep with a lot of top spin which tended to keep me from moving in. But he wasn't invincible either.

Totally agree. Self rated and appeal rated players are not skewing the ratings much at all, and they are not keeping well deserving players from getting bumped up. If you want to get bumped up, then win more matches at your current level by a wider margin. That’s how almost everyone who gets bumped does so, it’s really that simple.

People can disagree whether thousands of self rates and appeals playing out of level is "much at all" or not. It is like arguing whether a mile is a "long distance" or a "short distance."

The traditional view seems to be that playing people with higher dynamic ratings will help people get a bump not prevent it. That seemed to be the traditional wisdom leading to this blog:

Do stronger players in USTA League play singles or doubles? Interesting Tennis League Stats

In my work generating reports but also in questions I get in e-mail in response to my NTRP Ratings FAQ , a question I often get asked is if...

computerratings.blogspot.com

Self rates and appeal players playing below their level will have higher dynamic ratings than typical players so they would be people that help bump opponents not prevent it.

 

[Moon Shooter]

Another interesting blog by Schmke:

USTA Adult League Participation 2013 thru 2021 - Is the decline in unique players over?

Next for my year-end statistics and analysis is to look at participation numbers. I've done this analysis in past years , so I'm continuing...

computerratings.blogspot.com

He asks if the decline is over. I think that depends on the cause of the decline. If the cause of the decline was actions or inaction by USTA then I am not sure what we could point to that changed as far as how USTA is run. But I wonder if the decline has nothing to do with USTA and is really more about the baby boomer generation getting older.

"At an estimated 73 million, this generation is the second-largest age group after their children, the millennials, born from 1982 to 2000."

2020 Census Will Help Policymakers Prepare for the Incoming Wave of Aging Boomers

The 2020 Census will provide the most up-to-date information on the wave of an estimated 73 million aging baby boomers to policymakers.

www.census.gov

So about 130k of the 190k players were over 40. But we could expect that once you get over a certain age the number of people that want to play competitive rec tennis may start to drop off. So in 2022 the youngest of the millennials will start hitting 40. So if we just look at the population demographics overall this lull might be expected and the next 20 years should actually be looking really good for adult rec tennis participation if USTA stays the course.

Would staying the course be a good plan? I have to wonder if USTA can't do more to promote competitive league play for players under 40. I mean that such a small percentage of players are 18-39 is rather dismal. College players might understandably feel a bit of a let down going from college play - even club level play to playing with us older players. But I have to wonder if the possible requirement that all recent college tennis players rate themselves as a 5.0 is a deterrent. Especially for division 3 college players who were already just playing tennis for fun. I don't want to inflate the significance of that. Because I have found that most players that played in college are excellent players and in any case would not mind playing really tough competition. I also think that many of the issues that make league tennis more attractive to older people are out of USTA's control. But I wonder what things are in their control. For example, what about the rule that makes it so no one under 18 can play USTA rec tennis? If both younger and older players got used to playing each other early on would that make the transition less jarring? Any other ideas about how USTA could bump the number of young people playing in their leagues?

 

[schmke]

Another interesting blog by Schmke:

USTA Adult League Participation 2013 thru 2021 - Is the decline in unique players over?

Next for my year-end statistics and analysis is to look at participation numbers. I've done this analysis in past years , so I'm continuing...

computerratings.blogspot.com

He asks if the decline is over. I think that depends on the cause of the decline. If the cause of the decline was actions or inaction by USTA then I am not sure what we could point to that changed as far as how USTA is run. But I wonder if the decline has nothing to do with USTA and is really more about the baby boomer generation getting older.

FWIW, I do not think the decline is over. I had to combine 2020 and 2021 to get the slight uptick in participation, but as I showed, looking at either year individually the decline from 2019 would be precipitous. Obviously, there were other factors involved and even if one argues 2021 was a "complete" year with pretty much every area having the advancing leagues played and there was still around a 14% decline from 2019, it is understandable that some players were still reluctant to come back. But 14% is a big deficit to make up to just get back to status quo with 2019.

The challenge is that if you say that 2021 being 14% down from 2019 is COVID related (and I'm sure it was), COVID likely has a greater impact on those 50+ than those under that age, and given the aging USTA League participant population, it will require a more significant growth from the under 40 crowd to replace those that age out.

For example, in 2019 there were about 55K that played 55+ and there has been slight growth the past few years as more players age into 55+ or begin league play in their older years than are aging out. But if even an incremental 5% of these players don't come back to league play due to COVID over concerns or due to losing a battle with COVID the modest growth becomes a decline. Similarly, 2019 saw over 130K players in 40+, more than half of the total that played league at all. If an incremental 5% of these players don't come back, that is a 6,500 decline that has to be made up from new under 40 players, and 18+ was already losing a few thousand a year.