Is UTR Power 6 a Good Yardstick?

Tennis Sam

Rookie
Is the UTR Power 6 the best way to measure the quality of a team?

Here's how UTR voted in the latest USTA poll:
https://twitter.com/UniversalTennis/status/832221350123143170

A couple of points:
  • TCU has the second highest power 6. They have a 3-3 record overall with losses to Northwestern, Columbia, and Utah State. They aren't in Bobby's, Granger's, or TRN's top 25 (the other three voters on the USTA panel that published their ballots). I don't care how good their roster is, at least at this point in the season it's hard to justify them being the second best team in the country.
  • UVA, who has an absolute beating stick of a roster, has a power 6 that is 3 full points higher than the next best team in the country. I admit that they are really good, but they have already played two tight 4-3 matches against Kentucky and Vanderbilt. Those two team's power 6 numbers are 5 and 7 points lower than Virginia, and Vandy isn't even in UTR's top 25.
If the best team in the country is the team with the best roster, I think we can already coronate the champion. I doubt UVA's power 6 will fluctuate that much over the course of the season. Also TCU doesn't play again until next Saturday, so I doubt their power 6 will change much either. It'll be nuts if a team not good enough to qualify for the Indoors is still #2 next week.
 
Last edited:

jcgatennismom

Hall of Fame
Is the UTR Power 6 the best way to measure the quality of a team?
H.

No, the UTR Power 6 is definitely not the best measure of a team's ranking. It is useful to compare the relative strengths of conferences or to get a general idea if a recruit is a good fit for a team or conference.
Here's some reasons why the Power 6 might not be the best reflection of a team's strength
1) UTR was originally created for users to find competitive matches. Its rankings predict which matches should be competitive, not who will win. Suppose there were two players that both played 5 matches vs UTR 13.5 players. If Player A won 4 matches in 3 sets but lost the fifth 0, 2 that player would be rated lower than Player B who lost 4 matches in 3 sets but won one match 0,2. Now which player would a college coach put in his lineup, and which would he sit on the bench? I see players that are ranked .5 above their highest win because they have a few dominant wins (allow 4 games or less) and then I see players who are ranked .75 below their highest win and .5 below their hardest 3 set loss. Thes latter are guys that play almost every match close; they have a high win % but they are penalized because they dont soundly beat lower ranked guys-maybe they lose focus during the easier matches. UTR gives a lot of weight to dominant wins. However in college a win is a win and a loss is a loss; in UTR there is not much difference if any between a 3 set win and a 3 set loss.
2) The top 6 players on UTR may not even be the players the coach will play. I checked some teams I follow; for one the top UTR ranked player was playing 3 and the line 4 player actually had the 8th highest UTR but he is winning 60% of his matches at line 4
3) Freshmen could be considerably under or overrated on UTR. Freshmen who were US juniors tend to be overrated but drop over time. If I look at the current US freshmen, probably 80% are .5 lower than their UTR rating when they played Kzoo in August. International players who did not play on the circuit or junior ITFs may be under rated at the start of the season.
4) It also depends on how the Power 6 rating is split among the players .e.g a team with a 75 Power UTR could have two 14.5s and four 11.5s, The team could win 1 and 2 but lose the rest of the lines. Another Power 75 team could have two 13s, two 12.5s, and two 12s. A team can have a high Power UTR but if it is top heavy, it could lose the bottom lines.
5) Unfinished matches are not included in UTR. However, the coach is going to remember and play the guys who were leading and close to winning when the match was called. Those may be the guys who show up in the lineup even though they are not in the top 6 for UTR. Maybe one of the UTR top 6 is a guy who lost in 3 sets and still has a good UTR rating but cant close matches.

From the UTR website:
How are UTRs calculated?
UTRs are result-based and are generally calculated using a player’s last 30 matches within the last 12 months. The main factors are percentage of games won and opponent’s rating.
Therefore similar to my example in 1) a player who beat a 13 1,3 and then lost to another 13 in 3 sets would have a higher rating than a player who beat two 13s in 3 sets because the first player would have a higher % of games won even tho that player was 1-1 and the 2nd 2-0 as far as wins vs 13s. UTR penalizes players who pace themselves, e.g. players in a 128 Circuit Qualifier or a junior in a large draw tournament or even college players playing a doubleheader. In those situations, players may not run down every ball; they may use early rounds as a warm up.
 
If the best team in the country is the team with the best roster, I think we can already coronate the champion. I doubt UVA's power 6 will fluctuate that much over the course of the season.

Yes, UVA did have the best Power 6 Rating for the past 2 couple years. Altho they weren't top ranked, having the best Power 6 didn't turn out too bad for them.
 
No, the UTR Power 6 is definitely not the best measure of a team's ranking.

Thank you for all your comments and observations (which we take as suggestions for future improvements).

UTR is not meant to be a ranking. However, we do believe that it is a "Good Yardstick" when comparing teams -- especially when the top 6 players are in the lineup. And when the top players are not in the lineup, we have a Team Compare tool that can be used. The Power 6 Rating adjusts when players are moved on and off the top six playing positions.

See the College Tennis Today tweet at: https://twitter.com/College10s2day/status/832361093255282688

Team UTR works hard each week to come out with new enhancements to improve our rating system. You might have noticed our announcement yesterday. We have been working on a new algorithm for several months, and plan to release it this coming week. With the improved singles algorithm, comes a new doubles UTR. There are some that may believe we are not the best today, but we've working very hard to be there tomorrow.
 

jcgatennismom

Hall of Fame
Thank you for all your comments and observations (which we take as suggestions for future improvements).

UTR is not meant to be a ranking. However, we do believe that it is a "Good Yardstick" when comparing teams -- especially when the top 6 players are in the lineup. And when the top players are not in the lineup, we have a Team Compare tool that can be used. The Power 6 Rating adjusts when players are moved on and off the top six playing positions.

Team UTR works hard each week to come out with new enhancements to improve our rating system. You might have noticed our announcement yesterday. We have been working on a new algorithm for several months, and plan to release it this coming week. With the improved singles algorithm, comes a new doubles UTR. There are some that may believe we are not the best today, but we've working very hard to be there tomorrow.

We appreciate all the work UTR has done to come up with the only system that can compare college players, pros, and juniors from any country in the world. However, as far as evaluating teams, some of the ranking/rating system are done by people who are voting-people who know who is injured, who know which teams have been playing indoors vs outdoors, who almost beat a ranked player the week before in an unfinished match and other advantages/disadvantages that would not feed into a computer algorithm. I probably should have said UTR is not the best measure if it is compared to poll conducted by tennis experts, but it is probably the best or one of the best computerized systems; it is a fairly accurate rating system especially in terms of individual college players, but human rating systems have additional knowledge which makes them more accurate in evaluating teams. The people who vote can take the info UTR supplies and tweak it for those other items. I didnt realize about the team compare tool.

I am a monthly Premium Plus subscriber and use UTR for both junior and college match analysis so I obviously value the system-right now maybe it is a little better evaluating players than teams. It is a lot better predictor than many ranking/rating systems like USTA national rankings, but all systems can use some tweaking. I would say 80% of ratings or more seem reasonable but then there are outliers that dont make sense-players who seem .5 or more over or under rated. It will be interesting if some of those outliers look different after the new algorithm
 
We appreciate all the work UTR has done to come up with the only system that can compare college players, pros, and juniors from any country in the world. ... but all systems can use some tweaking. I would say 80% of ratings or more seem reasonable but then there are outliers that dont make sense-players who seem .5 or more over or under rated. It will be interesting if some of those outliers look different after the new algorithm

Thank you for your kind words and support as a Premium+ UTR subscriber. We are very excited about our new algorithm. So are the 50+ coaches that have already tested it against our existing system. This is the first major algorithm change we have made to our system. And the new algorithm framework allows us to make continuous rating improvements, which our team has committed to make on an ongoing basis.

We understand that there are "tennis experts" that will do a great job of providing input for team ranking polls. For everyone else, we invite them to try out the UTR system.
 

Tennis Sam

Rookie
Yes, 3 of their top 6 are not currently playing. Looks like we have some work to do.

Good point. Is it possible to change your votes to base them on the currently active roster? There are only two months until the championships. What if those players don't get better before the end of the season? They each have enough play that their UTR's won't change much and that'll keep TCU very high on your list no matter how the team performs.

Yes, UVA did have the best Power 6 Rating for the past 2 couple years. Altho they weren't top ranked, having the best Power 6 didn't turn out too bad for them.

I agree that UVA is the gold standard in men's tennis, but I think you missed my point. Even if UNC beats UVA today (like they did last year at the Indoors) or if the winner of Ohio State/Wake were to beat UVA in tomorrow's final, there is no way their individual UTR's will drop enough to displace Virginia from having the top Power 6. I'd be the first to say they still have the best roster, but I think one of those other teams would be more deserving of being called the #1 team.
 
Good point. Is it possible to change your votes to base them on the currently active roster?

A manual adjustment to the default Power 6 Rating for a team is something we are looking into, when it comes time to submit weekly votes for the UTSA Top 25 Rankings.

The current "automated" Power 6 Rating does not have the ability to determine if a team's posted roster is a reflection of the current team's playing roster. (The Power 6 Rating graphic you reference, is a data driven graphic that should be available to UTR Premium+ subscribers in the near future.) A team could have injured or ineligible players listed on their UTR team roster. Or, a coach could decide to place a player that is below the top 6 rated players into the playing roster for any given match. Again, this is why we have the Team Compare tool, in order to compare lineups prior to the match being played.
 

Tennis Sam

Rookie
A manual adjustment to the default Power 6 Rating for a team is something we are looking into, when it comes time to submit weekly votes for the UTSA Top 25 Rankings.

Without doing something manual for every team, could you just check the last time a player on the roster played a match? If the player hasn't played in the last month, for example, just don't count him/her in the adjusted Power 6 that you use to vote. Just a thought.
 
Is the UTR Power 6 the best way to measure the quality of a team?

What the UTR Power 6 Rating does, that team rankings do not provide, is the ability to group teams based on their level of play. Here is the above mentioned Week 2 Top 25...


C4yjT95VMAANPQd.jpg



Here is how the UTR Power 6 Rating did at the 2017 ITA National Men's Team Indoor Championships.


C5KRanAUMAALgUG.jpg
 
Last edited:

Tennis Sam

Rookie

You got me. I'm embarrassed I even asked the question.

But wait. Didn't ALL the systems have Virginia rated higher than each of their opponents?

What about South Florida beating Duke? What about Wisconsin beating San Diego?
(SPOILER: each had a lower Power 6 but were ranked higher by the other systems)

I suspect if you made that cool looking bracket for each of the different systems, the Power 6 wouldn't be better than ANY of the other systems.
(SPOILER: it's not)
 

Tennis Sam

Rookie
Revisiting this for the 2018 season:
  • The Tar Heels (10-0) just won the national Indoors and are #1 women's team in the ITA and USTA polls. #5 by Power 6.
  • TCU (5-2) is #18 men's team in both the ITA and USTA polls, and they didn't qualify for the national Indoors. #1 by Power 6.
TCU has a strong squad and may perform well moving forward, but they are definitely not worthy of a #1 ranking at this point in time. Who should be the #1 men's team is debatable, but right now no case can be made for anyone but UNC to be the #1 women's program.
 

JLyon

Hall of Fame
Revisiting this for the 2018 season:
  • The Tar Heels (10-0) just won the national Indoors and are #1 women's team in the ITA and USTA polls. #5 by Power 6.
  • TCU (5-2) is #18 men's team in both the ITA and USTA polls, and they didn't qualify for the national Indoors. #1 by Power 6.
TCU has a strong squad and may perform well moving forward, but they are definitely not worthy of a #1 ranking at this point in time. Who should be the #1 men's team is debatable, but right now no case can be made for anyone but UNC to be the #1 women's program.
Again until UTR can shift players around then number may not be accurate and when you have injuries to 3 of your top 6 players that is huge.
 
Top