ATX Tennis
Rookie
Sorry if this is readily accessible...I couldn’t find it. When is the last date that this year’s matches will count towards end of year ratings? Usually it’s some date in mid-late October. Thanks!
End of year ratings question
- Thread starter ATX Tennis
- Start date
schmke
Legend
Depends how you do, depends how your team does.
Even teams that make semis and/or win don't have every player bumped up, but Nationals team members certainly do get bumped up at a higher rate than the rest of the players.
Thanks. Guess we'll see in a couple of weeks.
If you just can't wait, I know a guy who does estimated DNTRP reports that are pretty reliable:
Example Individual Estimated Dynamic Rating Report
Below is an example individual report for a player's 2016 season. As of November 2017, this report format has been updated to include more ...
computerratings.blogspot.com
They're actually useful for a lot more than simply your dynamic rating, and I thought what I received in the past was well worth the cost.
OnTheLine
Hall of Fame
It is a computer. Basically an algorithm that spits out a number.
At the end of the year at the national level there may be some tweaks where adjustments are made based on how certain sections performed at nationals.
(e.g. if your particular section performed very well in post season play the ratings of that section in general may be pushed a little higher, if the section as a whole did poorly post season then the entire section may get a downward push)
But it is not an actual human looking at an individual record and making a determination, except in perhaps some rare cases (like people who stood out in post season play and likely to get double-bumped)
CiscoPC600
Hall of Fame
Haven't seen as much talk this year about the end of yer ratings update. If last year is any indication, aren't we a week or so away from the end of year rating change?
CiscoPC600
Hall of Fame
Last year it was released November 20ish, I thought..
schmke
Legend
Nope, 11/28, starting around 7pm PST.
CiscoPC600
Hall of Fame
Got it. I hope the 11/10 match does count since it was a good result for me haha.
MaxTennis
Professional
schmke
Legend
I don't think Open tournaments count, although I think it may be at a section's discretion.
Last edited:
schmke
Legend
Yeah, it is generally safe to assume the Monday after Thanksgiving, sometimes it stretches into the Tuesday or Wednesday depending on when Thanksgiving falls and if the USTA gets it done in time.
MaxTennis
Professional
winchestervatennis
Hall of Fame
I get what you’re saying, but if it were that simple, the ratings should be ready by today since it looks like all nationals are over but mixed. They’ve been doing it long enough that whatever adjustments are made for the sections that do well or poorly can easily be run through a computer calculation. So why would it take 3 more weeks if some level of human looking at them doesn’t take place. Methinks lists are shared at some level, maybe local, for input.
schmke
Legend
Well, I do have my year-end calculations done! Preliminary ones anyway.
So yeah, the USTA "should" be able to do it sooner too, but I think they have some built in review cycles to make sure nothing has run amuck, and I think there is some opportunity for regional adjustments after looking at what the computer spits out. Certainly some years there have been a lot of bumps up over all or in different regions that I think could only be explained by manual review and adjustment.
ej28405
New User
Just curious... Does mixed or combo count towards year end rating? And what about tournament results over the next couple weeks? Do they get factored into this or next years rating? Thanks!
schmke
Legend
Mixed only counts if you play only Mixed. Otherwise it is just your Adult league matches (and those leagues that your section opts in to including). Combo almost never counts, although again, it is something a section can opt to include. I think tournaments played after 11/10 won't count for 2019.
Vox Rationis
Professional
This is from an older document but provides some useful guidance on how postseason matches are weighted (20% for 1 postseason match, 40% for 2 postseason matches, 50% for 3 or more postseason matches): http://assets.usta.com/assets/558/Microsites/dps/usta_master/usta/doc/content/doc_90_106.pdf
Vox Rationis
Professional
Thanks. Funny enough my two matches at districts are already 40% of my total rated matches this year. The sad life of a 4.5 in a lot of areas
schmke
Legend
I believe elements of it are still accurate, but it was over 10 years ago and some things have changed.
More info on how the Nationals Benchmark ratings were used to calibrate ratings across Sections. From 12 years ago, so much could have changed -- I don't think they use verifiers anymore, but do think they assign greater weight to the rating of those who play the semis and finals at Nationals.
superseniorsrock.blogspot.com
BENCHMARK PLAYER RATINGS (how are they determined?)
By: Bob Greene USPTA, USTA Certified Verifier & Chairman of USTA NTRP Computer Sub-Committee
Annually each Section of the USTA will send up to twenty teams at all levels and genders to USA League National Championship Events. Those teams will play against each other in four flights of four or five teams in each flight. The draw is done at random. One event may be Florida, Texas, New England and Hawaii ... the next Eastern, Southern, ******* and Northern California.
Before and during these matches, no less than four of the most experienced NTRP Verifiers from different areas of the country will research the players match result history, multi-year rating history and player profile information. They will then observe the players competing against several different teams over a period of three days. All match results are entered into the NTRP Computer during the events.
The Verifiers are observing and are more specifically looking for lopsided match results, disparity of level between doubles partners and player improvement over the course of a season. The players who emerge from their respective flights to the semifinal and final rounds are given "absolute ratings". That rating is a number that is static for the purpose of comparison against other players.
These "Benchmark Ratings" are entered into the NTRP Computer and the computer program is run calculating ratings for all of the players who competed at the event. Although the NTRP Computer has an excellent track record of being correct, the National Verifiers makes a few adjustments based on reasons stated above. All of these players are National Benchmarks and their Ratings are deemed not changeable by regulation.
These National Benchmark Ratings are entered into the NTRP Computer and will filter down in each respective USTA Section and all players competing in NTRP audited and regulated venues will receive a rating if they played two or more matches. The primary goal and purpose of this methodology is to create and maintain uniformity in ratings on a nationwide basis. No matter what the picture appears to be from the bottom looking up, it is painted from the top down.
Benchmark Player Ratings - (how are they determined?)
BENCHMARK PLAYER RATINGS (how are they determined?) By: Bob Greene USPTA, USTA Certified Verifier & Chairman of USTA NTRP Computer Sub-Commi...
superseniorsrock.blogspot.com
BENCHMARK PLAYER RATINGS (how are they determined?)
By: Bob Greene USPTA, USTA Certified Verifier & Chairman of USTA NTRP Computer Sub-Committee
Annually each Section of the USTA will send up to twenty teams at all levels and genders to USA League National Championship Events. Those teams will play against each other in four flights of four or five teams in each flight. The draw is done at random. One event may be Florida, Texas, New England and Hawaii ... the next Eastern, Southern, ******* and Northern California.
Before and during these matches, no less than four of the most experienced NTRP Verifiers from different areas of the country will research the players match result history, multi-year rating history and player profile information. They will then observe the players competing against several different teams over a period of three days. All match results are entered into the NTRP Computer during the events.
The Verifiers are observing and are more specifically looking for lopsided match results, disparity of level between doubles partners and player improvement over the course of a season. The players who emerge from their respective flights to the semifinal and final rounds are given "absolute ratings". That rating is a number that is static for the purpose of comparison against other players.
These "Benchmark Ratings" are entered into the NTRP Computer and the computer program is run calculating ratings for all of the players who competed at the event. Although the NTRP Computer has an excellent track record of being correct, the National Verifiers makes a few adjustments based on reasons stated above. All of these players are National Benchmarks and their Ratings are deemed not changeable by regulation.
These National Benchmark Ratings are entered into the NTRP Computer and will filter down in each respective USTA Section and all players competing in NTRP audited and regulated venues will receive a rating if they played two or more matches. The primary goal and purpose of this methodology is to create and maintain uniformity in ratings on a nationwide basis. No matter what the picture appears to be from the bottom looking up, it is painted from the top down.
If the DQ zone has stayed the same I'm surprised at the actual # of DQs that happens. For a 3.0S > 3.30 and 3.5S >3.80.
Here is some analysis I shared with my 3.5 ladies team this week, based off of TennisRecord data (which not only is wildly inaccurate, but has been down for the last couple of days!):
In anticipation of the new ratings for 2020 (expected to be released Dec 2), I took a look at the 2018 40+ 3.5W National Champion team to see how many of them got bumped for the following season. The champs from PacNW had 18 ladies on the roster, including a couple of 3.0 players.
All bumped to 4.0 for 2019, except players with these records and TR ratings:
14-8; 3.30
11-5; 3.29
17-12; 3.23
11-5; 3.17
9-3; 2.95
1-1; 2.79
1-5; no rating
3-4; no rating
Everyone above 3.30 bumped, plus a 3.24 lady (who was just 14-6, 5-4 in postseason). Overall, just 10 of 18 bumped to 4.0.
If 2018 is any indication, then everyone on our team that is rated 3.30 and lower may be safe from bump up:
30-28; 3.30
36-11; 3.28
41-26; 3.23
36-32; 3.20
6-2; 3.19
64-26; 3.16
11-4; 3.07
Those of you that have estimated TR ratings above 3.30 should brace yourselves for the possibility of getting bumped....
Perhaps 2018 was an anomaly for 3.5 bump ups in MidAtlantic. 19 of the 20 guys on my 40+ 3.5 Nationals-winning team got bumped, including players with these TR stats:
3.28; 72-39 (2-4 in postseason)
3.28; 44-23 (2-0 in postseason)
3.32: 65-27 (2-1 in postseason)
3.32; 27-16 (didn’t play postseason)
3.36; 102-45 (3-3 in postseason)
3.39; 51-43 (1-0 in postseason)
And the 2018 18+ 3.5W from MidAtlantic went just 1-3 at Nationals, but had all but four 3.5 ladies bump up (and two of them appealed UP anyway), including ladies with these TR stats:
3.29; 97-49 (6-4 in postseason)
3.33; 37-16 (1-0 in postseason)
3.37; 105-37 (7-3 in postseason)
3.40; 56-34 (4-2 in postseason)
Seems they really gave a lot of weight to making Nationals, at least in 2018 among the MidAtlantic 3.5 population.
My hunches:
(1) TR ratings are vastly off from @schmke's, and presumably, the actual dynamic ratings as calculated by the USTA. It isn't unusual for these estimates to be off by 0.25 to 0.40, which is huge spread!
(2) The TR ratings do not assign greater weight to postseason matches, as the USTA does (I believe @schmke does, if you order a year-end report from him)
(3) No one knows the additional weight the USTA assigns for Nationals semis/finals participants, and the resultant trickle-down effect.
(4) I suspect manual adjustments can be made at the Section level (just not to those that have played at Nationals semis/finals...seems those are "absolute" ratings that cannot be manually adjusted).
The document posted above says that a .29 difference equals 4 games. Another USTA document says that the expected score between 3.01 and 3.49 is 12 games. Isn't that enough data to estimate the (non-playoff adjusted) algorithm? It's just that and rolling averages, right? I'm sure the expected .48 spread is tighter at the marginal levels, but that can't be the source of all the error.
Ft.S
Semi-Pro
I do not think the issue with TR ratings is limited to the weight of post-season matches. Throughout the year, once my # of rated matches goes beyond 12, the difference in rating results, compared to the @schmke reports I receive once a quarter is stark. Only at the end of September their ratings matched after about 55+ matches.
I do not know where TR is differing, but its ratings are very misleading IMHO. Possibly their comparison tables are off a bit, but the cumulative effect on rating calculations is significant.
I agree, the algorithm should be fairly easy to replicate... IF one had the starting ratings for all players. But because this (hugely significant) data point is not made available, the sites and others cannot be accurate. Over time, it's possible to hone in closer to the actual dynamic ratings by looking at players on the fringes, to see which players actually bumped or not. But the postseason and particularly the Nationals weighting make it impossible for these sites to be accurate, agree?
schmke
Legend
I do incorporate year-end calculations and refine them more each year, with continued improvements in my accuracy. My ratings are still not perfect, but my experience and that of others that I hear from and many that post here, is that they are more accurate than the alternatives.
@leech hits the right things, notably #4. In some years there are clearly regional adjustments made and there is no way for an algorithm to proactively predict that. How this is done and filters throughout the section/district/area can vary from year to year. And this complicates the following year as getting accurate starting ratings is important to calculating accurate ratings in the new year. Which is in itself a big factor in why a system may not be accurate. Inaccuracies in year X automatically make it hard to be accurate in year X+1 as you don't have the right starting ratings. So a less accurate algorithm will continue to be less accurate for this reason alone.
schmke
Legend
Oaf. This sort of bump up/down rate would be wildly inconsistent with normal years in general. Either players in your area played very poorly this year, or TR is a tad inaccurate for your area. Few bump ups is not uncommon at higher levels, but only 7 up and 50 down at 3.5 is highly unusual. Who knows, perhaps everyone is managing their rating looking to form a super team at 3.0 and 3.5 next year ;-)
BallBag
Professional
I looked at another area and it was a pretty even distribution so its not their algorithm. I wonder if were gonna get an above average number of bump downs this year.
Disregarded A and S rated players - just looking at 2019 C rated Male players I see this for TR estimated ratings in my district. No way that many 4.5/4.0 guys are getting bumped down in Mid-Atlantic. Especially after the 18+ 4.5 guys won nationals.
4.5C - up=2 down=18
4.0C - up=3 down=26
3.5C - up=10 down=13
4.5C - up=2 down=18
4.0C - up=3 down=26
3.5C - up=10 down=13
I see TR is updated because my Natl matches from last weekend are in....ranking not yet calculated. Anxious to see that.
