#### Moon Shooter

##### Semi-Pro

Edit: this thread is for anyone that wants to talk about or analyze data to see if it supports various opinions and also to think about what sort of data might prove or disprove certain opinions.

In this post I talked about how I would like to see certain data. As it turns out Schmke has collected quite a bit of data along the lines I was looking for.

This shows the year end ratings by his estimated dynamic rating every tenth of a point:

So I am not sure if he rounds up or down and I don't have the actual numbers but in any case just by eyeballing it it seems that the lower 2 fifths of 4.5 players seem to have more players than the upper 3 fifths of 4.5 players.

This would lead me to think the median 4.5 player should be about a 4.17 or so. Maybe the average 4.5 player rating would be about a 4.20.

Now these numbers are the dynamic ratings of people at the end of the year. So there may be some players that are outside these 5/10ths that are also 4.5 level players. But again since it is a level at the upper 10% of players you would think those 4.5 players that have a dynamic rating below 4.01 would greatly outnumber those with a dynamic rating above 4.50. At least all things being equal.

What are some things that might not make all things equal?

1) Appeals. Now if appeals were solely based on a persons honest judgment of their level and not to manipulate ratings so they could win we would likely expect about the same number of appeal ups as appeal downs. If the margins to appeal up are the same as the margins to appeal down then in the 4.5 level we would expect appeals process to lower both the median and the likely the mean rating. That is because there are just many more players that are say 3.93-4.01 than there are players that are 4.51-4.59. Also when we consider that most people tend to over rate their own abilities (such as most people think they are above average intelligence) we might even think that if the appeals process was really just for people honestly trying to correct the ratings it would cause the median and average to drop even lower. So maybe the median would be 4.15 and the average 4.19 or something.

But in this blog he found this regarding 4.5 players:

"First, I took a look at the average dynamic rating I calculated for all players that ended the 2013 year rated a 4.5 and found it to be

Now that is surprising. How can the average be so much higher than what we would think based on the relative number of people in this level? I mean you would think the average player is a 4.5 player if the average rating in that level happens to be 2.5 hundredths of a point away from the middle of the level.

Here are some thoughts:

A) Computer rated players shouldn't be causing this. Because computer rated players would seem to be as likely to go down as up when we are at this level and given the age of the average 4.5 level player. But I could be wrong.

B) People are appealing down much more than they appeal up. But as I explain in 1 above if the appeal process is being used honestly it should result in lowering the average rating of a player because there are simply many more players below this level then above it and human nature is to overestimate our ability not underestimate it. The only reason the appeals process would work to raise the average rating of a 4.5 player would be if they were appealing down in order to manipulate their rating. If someone rated down due to an injury and that injury in fact meant they were no longer a 5.0 player they would not have a year end dynamic rating pushing the 4.5 rating higher. Remember the first chart was the number of players at each .1 rating at the end of the year.

C) self rates are out of level and they are done to manipulate the rating. Again if people were just making honest mistakes in self rating you would think far more people below 4.5 would think they are 4.5 rather than 5.0s mistakenly thinking they are 4.5. Again this is because 1)there are far more people below 4.5 to miscalculate and 2) human nature tends to lead us to over estimate our ability rather than under estimate our ability. So if the self rating is pulling the anticipated average rating higher then the only cause I can think of is rating manipulation.

If I am understanding the data correctly then it seems the appeals and or self rating system are significantly distorting USTA leagues. I would be interested in any other thoughts on this data and where my analysis may be going wrong.

In this post I talked about how I would like to see certain data. As it turns out Schmke has collected quite a bit of data along the lines I was looking for.

This shows the year end ratings by his estimated dynamic rating every tenth of a point:

### Interesting USTA League Stats - Histogram of NTRP ratings down to the tenth

Next in my series of posts on interesting stats from the data I collect for the Estimated Dynamic NTRP Rating reports I generate, I took...

computerratings.blogspot.com

So I am not sure if he rounds up or down and I don't have the actual numbers but in any case just by eyeballing it it seems that the lower 2 fifths of 4.5 players seem to have more players than the upper 3 fifths of 4.5 players.

This would lead me to think the median 4.5 player should be about a 4.17 or so. Maybe the average 4.5 player rating would be about a 4.20.

Now these numbers are the dynamic ratings of people at the end of the year. So there may be some players that are outside these 5/10ths that are also 4.5 level players. But again since it is a level at the upper 10% of players you would think those 4.5 players that have a dynamic rating below 4.01 would greatly outnumber those with a dynamic rating above 4.50. At least all things being equal.

What are some things that might not make all things equal?

1) Appeals. Now if appeals were solely based on a persons honest judgment of their level and not to manipulate ratings so they could win we would likely expect about the same number of appeal ups as appeal downs. If the margins to appeal up are the same as the margins to appeal down then in the 4.5 level we would expect appeals process to lower both the median and the likely the mean rating. That is because there are just many more players that are say 3.93-4.01 than there are players that are 4.51-4.59. Also when we consider that most people tend to over rate their own abilities (such as most people think they are above average intelligence) we might even think that if the appeals process was really just for people honestly trying to correct the ratings it would cause the median and average to drop even lower. So maybe the median would be 4.15 and the average 4.19 or something.

But in this blog he found this regarding 4.5 players:

"First, I took a look at the average dynamic rating I calculated for all players that ended the 2013 year rated a 4.5 and found it to be

**4.23**."### More interesting USTA League stats - Comparing the 40 & over and 18 & over divisions

In 2013, the USTA introduced new age divisions in the Adult leagues, a new 40 & over league to go along with the existing 18 & over league, ...

computerratings.blogspot.com

Now that is surprising. How can the average be so much higher than what we would think based on the relative number of people in this level? I mean you would think the average player is a 4.5 player if the average rating in that level happens to be 2.5 hundredths of a point away from the middle of the level.

Here are some thoughts:

A) Computer rated players shouldn't be causing this. Because computer rated players would seem to be as likely to go down as up when we are at this level and given the age of the average 4.5 level player. But I could be wrong.

B) People are appealing down much more than they appeal up. But as I explain in 1 above if the appeal process is being used honestly it should result in lowering the average rating of a player because there are simply many more players below this level then above it and human nature is to overestimate our ability not underestimate it. The only reason the appeals process would work to raise the average rating of a 4.5 player would be if they were appealing down in order to manipulate their rating. If someone rated down due to an injury and that injury in fact meant they were no longer a 5.0 player they would not have a year end dynamic rating pushing the 4.5 rating higher. Remember the first chart was the number of players at each .1 rating at the end of the year.

C) self rates are out of level and they are done to manipulate the rating. Again if people were just making honest mistakes in self rating you would think far more people below 4.5 would think they are 4.5 rather than 5.0s mistakenly thinking they are 4.5. Again this is because 1)there are far more people below 4.5 to miscalculate and 2) human nature tends to lead us to over estimate our ability rather than under estimate our ability. So if the self rating is pulling the anticipated average rating higher then the only cause I can think of is rating manipulation.

If I am understanding the data correctly then it seems the appeals and or self rating system are significantly distorting USTA leagues. I would be interested in any other thoughts on this data and where my analysis may be going wrong.

Last edited: