Can anyone explain the rationale for USTA considering and granting appeals of Early Start and End of Year ratings that allows players to be be bumped down. Don't they trust their own system? Doesn't this play into the hands of sandbaggers who would rather win than play at their "real" level?
Appeals to bump down - What's the rationale for granting?
- Thread starter Reddirt
- Start date
Mikeler, I understand that some appeals will not be granted, but I have heard that they will be granted if your dynamic NTRP is within a certain tolerance (i.e., Your early start rating was 4.05, but your current NTRP is 3.99).
If I am incorrect about how this is done, hopefully someone more knowledgeable than myself will clarify.
The appeals are automatic and are based on your actual DNTRP. If you're within 0.05 of the level (and meet other criteria), the system grants the appeal, so it's only for people who the USTA considers "right on the edge". It's not that they don't trust their own system, it's that they recognize that ANY system is going to be imperfect. This is a difficult thing for sandbaggers to manipulate because you would have to have the ability to control your own DNTRP to within 0.05 without knowing the exact formula or any opponent's actual DNTRP. That's not easy to do.
Cindysphinx
G.O.A.T.
Also, USTA decided a few years back that players are not eligible to appeal down if they have played more than a certain number of matches. I think the magic number is three.
schmke
Legend
I believe it is actually 5, and the other criteria is you can't be a benchmark and have the auto-appeal granted.
mikeler
Moderator
"Players with 5 or fewer matches used in the Year-End rating calculation"
JRB, Thanks for your input. Do you know what "other criteria" they might use?
I think some players can pretty well figure out when they are close to the bubble. Once they figure that out, a few underperforming results could tip the balance...
dizzlmcwizzl
Hall of Fame
The goal of the NTRP system is to provide a competitive atmosphere for recreational players.*
The USTA does not allow automatic appeals if you have 5 or more matches played, or if you were benchmarked. The reason they do this is that with the 5+ matches or through the benchmarking process the USTA thinks they have enough data to reasonably assure you that you will be competitive at the assigned level.
If you only played 3 or 4 matches, the USTA simply does not have enough data to believe a player close to the line will be competitive at the assigned level. In this case they leave it up to the player to decide where they would best fit by allowing them to appeal up, down or not at all. By allowing players to access the appeal process they are attempting to fit each player into the right spot.
* This all assumes the USTA is actually trying to make a good experience for all of us ... and not simply trying to make as much money from their product as possible.
The USTA does not allow automatic appeals if you have 5 or more matches played, or if you were benchmarked. The reason they do this is that with the 5+ matches or through the benchmarking process the USTA thinks they have enough data to reasonably assure you that you will be competitive at the assigned level.
If you only played 3 or 4 matches, the USTA simply does not have enough data to believe a player close to the line will be competitive at the assigned level. In this case they leave it up to the player to decide where they would best fit by allowing them to appeal up, down or not at all. By allowing players to access the appeal process they are attempting to fit each player into the right spot.
* This all assumes the USTA is actually trying to make a good experience for all of us ... and not simply trying to make as much money from their product as possible.
Thanks, dizzlmcwizzl. Do you know if the 5 match rule for appealing down is a published rule, or standard operating procedures? Also, when appealing the early-start ratings, must these 5 matches be from the same calendar year? In other words, can a player, with an extensive playing history at one level, appeal early-start ratings if they only played in 4 matches in the spring?
dizzlmcwizzl
Hall of Fame
The 5 match rule for the appeal of year end ratings is a published rule. In fact someone quoted it earlier. These matches must have occurred during the year that the year end rating is based upon.
Information about the early start ratings appeal process are less well known. My personal belief is that there is a person in each section responsible for all questions related to ratings in that section. I believe that these people have guidelines that they adhere to but that the appeal process for early start is less regimented than the year end, which is strictly computer based.
In my section the appeal of early start ratings is not automatic. We must e-mail a specific woman and she looks at our current dynamic rating and manually makes adjustments, if necessary, to our early start rating.
I have appealed early start ratings in past years and have gotten detailed responses that seemed to suggest that this person had some leeway in making early start appeal decisions.
For example, one year her response was something along the line of ... "Your early start rating is over the threshold for appeal and in looking at your playoff results I do not believe that after benchmark calculations occur you will be close enough to appeal at year end". This seems to suggest to me that if I had not done well in sectionals then she might have granted an appeal even though my early start was outside the range.
Of course I don't know for sure. We know a good bit about year end ratings and the dynamic rating. However, much less is known about the mechanics of appealing early start ratings.
dizzlmcwizzl
Hall of Fame
Oh, and to answer your question ... yes you can appeal. Everyone can appeal ...
If you played more than 5 matches or were benchmarked you will automatically lose your appeal regardless of what your year end rating is.
also, if you played so well you are outside of the range ... you will lose your appeal. But everyone is entitled to ask.
BY THE WAY ... you seem very concerned with appealing an early start rating. Are you aware that year end ratings will be published on December 2nd ... about 5pm? Once they are published, early start ratings become pretty much meaningless unless you are already on a team that uses early start ratings.
mikeler
Moderator
December 2nd will be an interesting day for many.
Thanks again, dizzlmcwizzl. Despite my lack of clarity, you, in your earlier post, pretty well addressed my intended question which should have read "Is the same 5 match rule adhered to in granting early-start appeals."
I understand your puzzlement about my concern over early-start ratings. I must admit it is based on a particular instance of a player in my area being granted an early-start appeal to bump down (He played 4 matches in the spring). I am well familiar with this player's game, and feel he will likely trounce my singles player (in playoffs), even though I worry that my player, who is computer-rated, will be bumped up at EOY - maybe I shouldn't worry... Still, it is aggravating, especially when the team for which this player plays, has a history of recruiting from out of area to build the strongest team possible, and I have heard from another 4.5 player of having been approached by this team's captain about getting himself bumped down to 4.0 so that he can play for this 4.0 team. In addition, the bumped player in question has some very suspicious results (i.e., underperforming against lesser opponents and winning easily against stonger opponents). My team is in the playoffs for the first time in 4 years, and it has been a long, hard building process... Perhaps you can feel my pain...
The bottom line is that I could not understand how this player's appeal was granted, so my posts have been an attempt to try to understand the process. Thanks to you and all who have tried to educate me. I appreciate it.
I realize that this team and player may have outsmarted themselves with regard to EOY ratings, but his results suggest very careful management of ratings.
Adles
Rookie
ok, so silly question here: where on tennislink's byzantine website do the year end ratings get published?
schmke
Legend
First, they aren't published typically until the Monday after Thanksgiving (12/2 this year).
When they are published, you can do searches from the main USTA League page (http://tennislink.usta.com/Leagues/Common/Default.aspx, "Find NTRP Rating Info" box).
Adles
Rookie
Thanks, got it. I was wondering if they were published in a pdf doc in the same way that the early start ratings are. For fun bedtime reading...
chatt_town
Hall of Fame
As a computer programmer for the last 18 or 19 years this is what I don't understand about early ratings and appeals. If the computer deems you at early ratings to be bumped to say 4.5 and then someone manually goes in bumps it back to 4.0, isn't the computer going to come up with the same calculation at YE? For example the computer deems that 2+2 = 4 and then you manually change it for whatever reason to be 3. When the computer computes 2+2 on December 2nd, isn't it going to be 4 again? JRB? what's up bro? Can you explain a little more?
schmke
Legend
You are right, what you describe could happen. Someone could appeal an early start bump but then be bumped up at year-end.
But keep in mind that the "equation" for early start is not the same as it is for year-end. Even without appeals, players can and are bumped up/down for early start but go back to their original level at year-end. Or they aren't bumped up/down in early start but are at year-end.
This happens because early start is more or less just an indication of the dynamic rating on a certain date, while year-end ratings factor in not only matches played after the date, but leagues and tournaments that aren't part of the dynamic rating during the year as well as the benchmark calculations.
The short answer I would think is yes, but there are two things to consider. First, the nationals matches and the benchmarking process that is the difference between the ES and YE has an effect on everyone in a section, not just the players who played at nationals. I don't know how to quantify what that effect is, nor do I think even Schmke has a handle on that one. Second, and probably more importantly, the decisions should be made by someone looking at your DNTRP. If a sectional coordinator who is reviewing the ES appeal sees that, unless something significant happens, you are likely to be eligible for an automatic appeal at YE, then granting the ES appeal is really just granting the auto appeal early.
schmke
Legend
I took a look at the player in question and he was close to being bumped down to 4.0 at 2012 YE and was around the same point after the Spring season, so can see how he would have been close enough for an appeal down to be accepted.
That said, his Fall league results are suspicious and would fall into the potential sandbagger category. His results (all at 4.5) early in the year were more losses than wins and were in the range of 3.8-4.2 which looked normal/expected, but his 4 Fall league results have gone 3.6, 3.6, 3.4, 3.3, the first two in a 4.5 league the last 2 in the 4.0 league he was able to get in after appealing. It definitely looks like he is not playing at the level he can and potentially doing so in order to ensure his appeal down sticks at year-end.
This is backed up by his two matches in November, after the 2013 year ended (10/27) where he has now won a 4.5 match and thumped someone in a 4.0 match. IMHO, this appears to go beyond just suspicious.
Note also that his appeal down is slightly odd, as if you include the 2 Fall league matches played in November/December 2012, he actually had played 6 matches in the 2013 league year. According to the 5 or fewer rule, this should have made him ineligible to appeal down. So perhaps the 5 or fewer isn't a rule for early start appeals, only year-end? Or even though the Fall league counts for ratings, it has to be 5 or fewer Spring league matches? My guess is that the reason he was able to appeal is that early start appeals are manual and at the discretion of the LC and don't have to abide by the 5 or fewer rule.
Last edited:
schmke
Legend
Hey now, I'm working on it ...
Seriously, while I don't have something fully implemented for benchmark calculations, when I do reports I take a look at what I think the benchmark calculations will do and often can use this to predict/explain why a player with a dynamic rating below the bump threshold will/does get bumped.
--
http://computerratings.blogspot.com/search/label/tennis
Hey now, I'm working on it ...
Seriously, while I don't have something fully implemented for benchmark calculations, when I do reports I take a look at what I think the benchmark calculations will do and often can use this to predict/explain why a player with a dynamic rating below the bump threshold will/does get bumped.
--
http://computerratings.blogspot.com/search/label/tennis
Not meant as a slight, LOL. Your algorithm is remarkably accurate.
And remarkably fun. The ratings have been used for quite a bit of trash talking within my men's team.
Also, ES ratings come out before some of the sectionals are played too, so it isn't just nationals that isn't included.
GlennK
Rookie
Sure, they could get the bump again but this would be after including the next year's results. If the person is bumped down and does not do as well the following year, the algorithm will not give them the bump back up.
SunshineJS
New User
FYI...
the 5 match rule is no longer in effect. It's now a sliding scale which will determine how much leeway the auto appeal will have based on how many matches a given player has played.
the 5 match rule is no longer in effect. It's now a sliding scale which will determine how much leeway the auto appeal will have based on how many matches a given player has played.
mikeler
Moderator
Where did you see this info?
SunshineJS
New User
I work for the USTA...
Can't/won't give full details, but it is no longer the cut and dry .05 threshold and anything over 5 matches played is automatically denied.
schmke
Legend
I just saw it in a newsletter from my section. Blog with details coming shortly.
schmke
Legend
Take a look at http://computerratings.blogspot.com/2013/11/new-rules-for-usta-league-ntrp-rating.html
The quick summary is that the appeal range is now based on the number of matches played with a couple special cases for seniors. If you played only 3 matches, the range is 0.10, 4-5 matches, 0.08, 6-7 matches, 0.06, 8-9 matches, 0.04, and 10 or more matches you can't appeal.
The quick summary is that the appeal range is now based on the number of matches played with a couple special cases for seniors. If you played only 3 matches, the range is 0.10, 4-5 matches, 0.08, 6-7 matches, 0.06, 8-9 matches, 0.04, and 10 or more matches you can't appeal.
Last edited:
schmke
Legend
SunshineJS, in what I received there was no mention of benchmark players not being allowed to auto-appeal down. Can you confirm this limitation is no longer there?
Now, someone who is a benchmark will likely have played more matches during the year and so at a minimum will have a lower allowed range or perhaps not even allowed to, and so this may address situations where a player shouldn't be allowed to auto-appeal down.
mikeler
Moderator
I'm definitely interested in this as a benchmarked self-rate with only 4 matches played.
A "benchmarked self-rate" is not a category under USTA definitions.
I interpret this to mean that you started the season as a self-rated player and played four matches, at least one of which was in post-season play. If this is the case, you were a self-rated player in 2013 and will be a benchmark player in 2014.
mikeler
Moderator
Correct. I played 2 local matches and 2 playoff matches.
schmke
Legend
From what was in the PNW newsletter then if it wasn't missing the benchmark exclusion, you should be able to auto-appeal down as long as you are within 0.08 of the bottom of the range.
mikeler
Moderator
I only played 4 matches but the guy I beat at Sectionals fell apart after 4 games. That one blowout win will likely skew my final rating above where I think it should be.
The rating calculations become more accurate when they have more data to use. Because of this, the people who play the most matches will have the most accurate ratings.
If someone played 10 matches and got killed in every match, they wont have to worry about appealing to get bumped down. They would simply get bumped down without any appeals.
schmke
Legend
Well said.
I think this is a more reasonable rationale than the previous. There was a matches played component before, but it seemed to be a binary decision that if you played more than 5 matches you couldn't appeal, and the range was fixed at 0.05 over.
Now if you only play a few matches, they are admitting that your rating may not be perfect with the limited data and so if you are a bit farther over, you can auto-appeal. But regardless of if you went to playoffs or not, if you've played 10 or more matches, they consider your rating pretty accurate and auto-appeals aren't allowed. And by graduating it by matches played they eliminate the binary decision.
Slightly off topic question but hopefully OK given that I feel like the OP's question has already been answered/explored sufficiently...
When trying to figure out how many matches I have towards the ratings calculation for the next year, how do I know when the cutoff is? I know which leagues/game types do and do not count, but I also heard that matches played after a certain date won't be factored in until next year and so they won't count towards bump ups/downs.
When trying to figure out how many matches I have towards the ratings calculation for the next year, how do I know when the cutoff is? I know which leagues/game types do and do not count, but I also heard that matches played after a certain date won't be factored in until next year and so they won't count towards bump ups/downs.
schmke
Legend
The "USTA year" is generally November to October, although the specific dates vary based on when weekends fall and when the last Nationals is held. This year the end of the 2013 year was 10/27.
schmke
Legend
You are correct for noting that players that appeal are subject to 3 strike DQs. But since she was close enough to appeal, her rating was lower than the threshold for strikes. So as long as she doesn't improve or have a string of above average matches, she is likely safe. But this is one of the risks of appealing down.
Maybe. Is she really a 3.5 or just a good 3.0? This is one of the safeguards of the appeal system. If you're just a nudge over the line and you think you should be at the lower level, go ahead and appeal, but you're subject to DQ so if you're lying to game the system and sandbag, you're going to get DQd.
keithfival
Professional
This must be true as someone here played 7 matches this year and got an auto appeal down granted.
schmke
Legend
See http://computerratings.blogspot.com/2013/11/new-rules-for-usta-league-ntrp-rating.html for a summary of the new rules.
mrpeterman
Banned
I didn't read the whole thread so this might have been mentioned but the reasons to bump down can be injury, age and inactivity. There are also a lack of 5.0 and 5.5 leagues around the country so the more advanced players have no where to play. Lots of former 5.0 and 5.5 players around here are playing in 4.5 leagues.
Some of the rising 4.0 players have no chance... but getting completely dominated has a way of motivating some players to get even better.
Some of the rising 4.0 players have no chance... but getting completely dominated has a way of motivating some players to get even better.
Similar threads
