TennisRecord EOY Accuracy for year end 2022

[coldstart]

Okay now that end of year data is published by USTA and updated in TennisRecord, let's see how well TR does in forecasting "bump up" or "bump down". TR publishes a "2022 Projected Rating" (which is not available on website anymore). I took a snapshot of all the TR year end projections and compared to USTA 2023 actual ratings.

Disclaimer: I don't work for TR.

I find TR data very helpful giving relative performance to other team members and leagues. As a USTA captain, it's good to see who we are playing against as the dynamic ratings reflect, in general, the strength of the players. I do see some discrepancies when it comes to single specialists vs doubles but overall TR is better than any other tool out there for updated information. Let's give them a round of applause for providing this info for free. My unpopular opinion is that it's ran by USTA shell company / ex-employee only because the site is a mystery with no contact information, the WHOIS domain is masked, etc.


The Results

• Overall score (bump up + bump down + stay same) / total ==> 84% correct.
• If TR had you as "bump up", probability it was correct for male/female: 64% / 76%
• If TR had you as "bump down, probability it was correct for male/female: 44% / 45%
• If TR has you as "same", probability it was correct for male/female: 93% / 92%
• The lower your rating, the higher probability the bump up/down was correct. For example, if you were a 2.5 female and TR said you were getting bumped up, 84% it was correct.

The Data in a Chart

The orange line means TR was correct for the level. Read the chart from the bottom up. For example, for Females that stayed "same", there were 35k women at 3.5 level.

@schmke if you want to slice and dice this data more, send me a PM and I'll send it to you. Also I will not do any more analysis on this data as I'm busy watching the World Cup.

 

[LOBALOT]

One reason I think the bump down accuracy is so low is I really think the USTA biases against bump downs so even if the statistics say a player should be sent to the minors I don't think they do it unless it is an obvious / extreme case.

 

[TennisOTM]

One reason I think the bump down accuracy is so low is I really think the USTA biases against bump downs so even if the statistics say a player should be sent to the minors I don't think they do it unless it is an obvious / extreme case.

Yeah the bump down prediction is pretty bad - less than 50% of those predicted to be bumped down were correct. You'd make money betting against their predicted bump-downs at even odds, yikes.

I think it has more to do with TR's estimates skewing too low, which is pretty well documented. I'd be very surprised if USTA just decided to keep players at level even if their year-end NTRP came out lower.

I wonder how much better these overall results would be if TR simply added 0.1 to every player's final rating result across the board?

Btw, nice work @coldstart and thanks for sharing this!

 

[Moveforwardalways]

You have to factor in some form of pretest probability too. For instance, more 2.5’s are going to be bumped up almost by definition every year.

 

[silverwyvern4]

Wow cool chart.
Anyone else notice UTR and Tennis Record haven't updated results from USTA in over 3 weeks?

 

[time_fly]

TR historically goes on hiatus around this time of year. It may be because they try to do some sort of annual maintenance or recalibration based on the bump data, or maybe it’s just run by Santa and he gets busy.

I have no explanation for UTR.

 

[jm1980]

I find TR data very helpful giving relative performance to other team members and leagues. As a USTA captain, it's good to see who we are playing against as the dynamic ratings reflect, in general, the strength of the players. I do see some discrepancies when it comes to single specialists vs doubles but overall TR is better than any other tool out there for updated information. Let's give them a round of applause for providing this info for free. My unpopular opinion is that it's ran by USTA shell company / ex-employee only because the site is a mystery with no contact information, the WHOIS domain is masked, etc.

The WHOIS on my domains are also masked, because screw putting my personal information out there for the world to see. A lot of registrars offer this at no extra charge, so why wouldn't you take it?