2012 Year End Ratings

[g4driver]

OrangePower,

Thanks for adding back to the discussion. I don't recall ever asking to have Single's Leagues count towards NTRP, but rather have all Sections count the same thing. I could very well be wrong. I haven't reread my own posts. I think I asked why don't Single's League's count towards an NTRP. But I didn't realize that all Sections don't have Single's Leagues.

As far as the USTA's NTRP algorithm works, if it were truly accurate system the benchmarks the USTA uses already in the Adult League would carry over to the other Leagues, including Mixed, Combo and Singles. The USTA already averages averages which a math no-no, so why the USTA doesn't follow accepted mathematical logic shows a flawed system. It is a combination of a bad algorithm and non-standardized inputs that flaws the USTA NTRP.

e.g. Some players only play Mixed and get a Mixed Exclusive Rating. They then play the following spring season starting with a "M" rating, but that rating seems to be the starting point for the new DNTRP. 1/2 of last year's rating and the first match of the new spring season = the new DNTRP, and the next matches will content to update the DNTRP as the old match in the spring "waterfalls" out of the equation. The problem is the first match is 1/2 of the new DNTRP. Tank that match and your DNTRP rating will drop. Win big and your DNTRP rating will rocket up. This is completely and utterly flawed math. But hey, that is the USTA. At least the USTA knows they have a problem. Why else would they have put out a survey about a new rating system? I am not alone telling the USTA their math is flawed. I got paid good money to write algorithms and while the USTA likes to think their NTRP algorithm is secret, parts of it have been leaked all over the internet.

[g4driver]

JRB,

I have read and reread your posts. Great I get it. Your ok with Sections "Doing their own thing", you can't come up with one example of how my idea hurts honest people and I don't need you to tell me to ask each section.

I am starting my local level, then state, then to the National USTA. It's called the chain of command, and I am marching up it with facts not opinion.

I simply asked you if you, username "JRB", if you could give me an example of how my idea hurt any honest player. You can't therefore I will continue my quest to fix what I do care about and have enough facts to at least prove to the USTA that their "Self-Rate" policy is being abused by a handful of players/Captains since the USTA relies on honesty from it's players/Captains.

It's great that you don't care. I never asked you to do any research. I've got a list of names and records that I will present to the SC USTA and I will asked some very direct questions in front of an audience. Part of the problem IMO, is there are too many people who have what I would call "NASA", or N/A on SA or low Situational Awareness. They don't have a clue where they are, or what is going on around them.

Not a dig at them. They just see a guy as a uncommonly good "3.5" player, when I see him for what he is - a cheater, at least a 4.0, and probably a 4.5. I care in principle and in practice and I've chosen to do something about it. I might fail in my goal, but I at least know I tried to argue my case, instead of standing on the sidelines like a coward complaining about something, yet choosing to do nothing.

Hope this makes my agenda crystal clear to you.

[Cindysphinx]

Let me ask you some questions, if I may.

You wrote:

2) The USTA develops and implements one National set of procedures regarding how, what, when and where grievances are filed, and how, what and who will rule on grievances. Standardized policy for vacating wins when players are DQed, and standardized punishment for known and willful violations of USTA rules regarding league play.

Question 1: Say Hawaii vacates all wins of a self-rate player who is DQ'd, and Mid-Atlantic only vacates the last win. How does this affect someone in Southern section? Why would a player in Hawaii care if Mid-Atlantic would have handled this differently? Either way, the player is not eligible for the post-season and is forced to play at a more appropriate and higher level, and that is mostly what National cares about.

Question 2: Say Mid-Atlantic, having a small budget, charges everyone who files a grievance $50. Say Southern, having a huge budget, charges nothing to file a grievance. Say Eastern, having a moderate budget, charges $100 to file a grievance with a refund if the grievance is upheld. What is the harm in allowing this local variation in grievance procedures?

You wrote:

a) those players playing in these leagues would be subject to strikes against them per current or revised USTA procedures. This solves the problem of sandbaggers at 4.5, who self-rate at 3.5, then bagel, 0&1, and 1-1 4.0C rated players.

Question 3: How is this different from the existing three strikes system for self-rated players? In your example, the self-rated player played only two matches and so could not have accumulated three strikes. If he beats down another 4.0 C-rated for a third strike, presumably the computer would have the results it needs to DQ him. How does your proposal differ from the existing system?

[catfish]

Originally Posted by g4driver

JRB,

I have read and reread your posts. Great I get it. Your ok with Sections "Doing their own thing", you can't come up with one example of how my idea hurts honest people and I don't need you to tell me to ask each section.

I am starting my local level, then state, then to the National USTA. It's called the chain of command, and I am marching up it with facts not opinion.

I simply asked you if you, username "JRB", if you could give me an example of how my idea hurt any honest player. You can't therefore I will continue my quest to fix what I do care about and have enough facts to at least prove to the USTA that their "Self-Rate" policy is being abused by a handful of players/Captains since the USTA relies on honesty from it's players/Captains.

It's great that you don't care. I never asked you to do any research. I've got a list of names and records that I will present to the SC USTA and I will asked some very direct questions in front of an audience. Part of the problem IMO, is there are too many people who have what I would call "NASA", or N/A on SA or low Situational Awareness. They don't have a clue where they are, or what is going on around them.

Not a dig at them. They just see a guy as a uncommonly good "3.5" player, when I see him for what he is - a cheater, at least a 4.0, and probably a 4.5. I care in principle and in practice and I've chosen to do something about it. I might fail in my goal, but I at least know I tried to argue my case, instead of standing on the sidelines like a coward complaining about something, yet choosing to do nothing.

Hope this makes my agenda crystal clear to you.

G4driver, it sounds like you put a lot of time and energy into this. I’m really curious as to why this is so important? I’ve played USTA league tennis for about 11 years now. In general, I find it a great way to have competitive matches in an organized setting. The opportunity to advance to post season championships and compete against players from other areas is also a nice perk. IMO, the NTRP rating system does a pretty darn good job considering the number of people involved. I know that there are sandbaggers and cheaters out there, but no matter what the system is they will find a way to sandbag and cheat. Some people are like that.

What I’ve never figured out is, why do people become so obsessed with changing the rating system and criticizing the USTA league program as a whole? Sometimes I wonder how many hours some people spend on such things. What drives you g4driver? I’m not trying to pick on you, but I see a lot of people like you and I don’t know where the get the time or energy.

[ian2]

Originally Posted by OrangePower

I don't get your logic. One the one hand, you say to count all same-gender matches, because all parties have a known DNTRP. But then you say you don't want to count mixed results. Why one and not the other?

Methinks whatever justification you're gonna come up with for not including mixed results could easily be extended to argue for not including combo results for example, which of course contradicts your first proposition.

OrangePower, my preference would be to include every "official" USTA match where each player involved had a known DNTRP before the match was played. The main reason I'm opposed to including mixed/combo results is that I don't trust the NTRP algorithm to come up with "equitable" ratings if everything was thrown in. As is, they can barely manage to keep up with the more homogeneous scenarios (same gender singles/doubles). The basic DNTRP algorithm is somewhat flawed (g4driver's post #61 in this thread touches on some of the reasons) but in the end, in my opinion, the algorithm comes up with "reasonable" outcomes most of the time for most of the players. I'm not at all certain this would continue to be the case if mixed/combo results were included.

On a somewhat related note, I'd like to see a move towards a more transparent and more information-rich system. I don't want to open a new can of worms here (or more likely, a dozen new cans) but personally, I'd like to:
1. know my exact DNTRP... ideally, in real-time; at the very least, once a year when year-end ratings are published
2. along with the overall rating, know my separate rating in singles, doubles, and mixed.

[ian2]

Originally Posted by catfish

G4driver, it sounds like you put a lot of time and energy into this. I’m really curious as to why this is so important? I’ve played USTA league tennis for about 11 years now. In general, I find it a great way to have competitive matches in an organized setting. The opportunity to advance to post season championships and compete against players from other areas is also a nice perk. IMO, the NTRP rating system does a pretty darn good job considering the number of people involved. I know that there are sandbaggers and cheaters out there, but no matter what the system is they will find a way to sandbag and cheat. Some people are like that.

What I’ve never figured out is, why do people become so obsessed with changing the rating system and criticizing the USTA league program as a whole? Sometimes I wonder how many hours some people spend on such things. What drives you g4driver? I’m not trying to pick on you, but I see a lot of people like you and I don’t know where the get the time or energy.

catfish, some of it is "professional"... In my case, I see an algorithm that works OK but (I think) could work better, and feel a compulsion to discuss. Some people are like that I do agree that overall, NTRP system works - despite some obvious algorithm problems, the necessity to "adjust" the ratings periodically (2009 en-masse bump-ups?), etc.

[catfish]

Originally Posted by ian2

catfish, some of it is "professional"... In my case, I see an algorithm that works OK but (I think) could work better, and feel a compulsion to discuss. Some people are like that I do agree that overall, NTRP system works - despite some obvious algorithm problems, the necessity to "adjust" the ratings periodically (2009 en-masse bump-ups?), etc.

Perhaps the algorithm hasn't changed too much because of costs involved. I think that not allowing benchmarked players to appeal their year end ratings was a good move. It ended some "team dynasties" that had been winning at their level for years. I know in my local leagues there are not near as many repeat winners.

I'm not so sure I like the en-masse bump-ups. Maybe I got used to what I think of as the "old ratings". But now, seems like everyone moved up one level except not many 4.5's moved to 5.0. I'm a 4.5, and don't especially want to get moved to 5.0 due to limited league play opportunities. But so many 4.0's moved up to 4.5 in the last couple of years that a lot of the match results are very lopsided. Many 4.5's kept their ratings and a whole slew of 4.0's moved up. There is a huge gap in the levels within the level.

[Islandtennis]

Very good insight about this subject catfish. I agree that the NTRP system is not perfect but overall it accomplishes what it intends to do except when people intentionally try to manipulate the system.

The intention is not rank everyone in a local area on a 1000 person "ladder", it is to give people of similar ability the opportunity to have a competitive match and to allow a 3.5 in GA to play 3.5 in CA and it be enjoyable. The system is doing this. Sure there are some 3.5 rated players who play at a 4.0 level and vice versa, but these are the minority and will usually get bumped up or down in time.

Everyone looking for an overhaul of the system needs to study how the system works at present. For instance, using Mixed or Combo for benchmark ratings (as mentioned a few posts up) would cause your 2011 ratings not to come out until Spring of 2012 (as Nationals are complete). Counting every match played would give you more data to use, but it would be flawed because of the different type of leagues in different areas.

For G4, Standardization would do little to make the ratings in your local area more accurate.

[SuperLotto]

When are the new NTRP's coming out? December 31st? If I played 14 3.5 level matches and only lost one...the first one of the season, will I get bumped up to 4.0?

[kylebarendrick]

Ratings usually come out around December 1. I say you have a good chance of being bumped, but you never know for sure.

[dizzlmcwizzl]

Originally Posted by SuperLotto

When are the new NTRP's coming out? December 31st? If I played 14 3.5 level matches and only lost one...the first one of the season, will I get bumped up to 4.0?

They always come out the Monday after thanksgiving about 5:00.

[bruce nissenbaum]

Originally Posted by SuperLotto

When are the new NTRP's coming out? December 31st? If I played 14 3.5 level matches and only lost one...the first one of the season, will I get bumped up to 4.0?

Seems like it should happen, but not necessarily. As I understand it, if the computer model predicted that you should have won all those matches easily (i.e. 6-1, 6-2) and your wins were real close (i.e. 6-4, 7-6), it is very possible that your DNTRP could actually decline and your EOY rating could remain at 3.5!

[OrangePower]

Originally Posted by ian2

OrangePower, my preference would be to include every "official" USTA match where each player involved had a known DNTRP before the match was played. The main reason I'm opposed to including mixed/combo results is that I don't trust the NTRP algorithm to come up with "equitable" ratings if everything was thrown in. As is, they can barely manage to keep up with the more homogeneous scenarios (same gender singles/doubles). The basic DNTRP algorithm is somewhat flawed (g4driver's post #61 in this thread touches on some of the reasons) but in the end, in my opinion, the algorithm comes up with "reasonable" outcomes most of the time for most of the players. I'm not at all certain this would continue to be the case if mixed/combo results were included.

Your intentions are good, but...

You say that you trust the algorithm in scenarios where the players are close to one another in DNTRP, but not when they are further apart. And as a result, you propose that all same gender singles/dubs should count, but combo/mixed should not.

The flaw is that you assume DNTRPs will be consistently closely grouped in singles/dubs, and not closely grouped in combo/mixed. That's probably a good generalization, but is not correct under multiple scenarios:

1. Players playing up in singles/dubs leagues: What if a 3.5 plays a 4.0 in a 4.0 singles league? Should that count despite the disparity in DNTRP?

2. Players playing up in combo leagues: Say two 4.0s are playing against two opponents who are also both 4.0s in a 8.5 combo match. Shouldn't this count using your reasoning?

3. The arbitrary nature of the cutoff between rating levels: DNTRPs are calculated using a continuous function. Breaking this up into levels introduces an artificial discontinuity. This leads to situations where players at different levels can be closer in DNTRP than players at the same level. For example, at 4.0 dubs you might end up with four 4.0s having DNTRP of {3.51, 3.63, 3.75, 3.99}. At 8.5 combo you could have two 4.0s and two 4.5s with DNTRP of {3.99, 3.99, 4.01, 4.01}. The DNTRPs in the latter case are much more tightly clustered than in the former, despite the fact that there is a mix of rating levels.

I'm sure one could come up with another layer of rules to include/exclude scores under scenarios depending on DNTRP spread and so on. But then this is getting much more complicated and will in turn introduce more issues.

Originally Posted by ian2

On a somewhat related note, I'd like to see a move towards a more transparent and more information-rich system. I don't want to open a new can of worms here (or more likely, a dozen new cans) but personally, I'd like to:
1. know my exact DNTRP... ideally, in real-time; at the very least, once a year when year-end ratings are published
2. along with the overall rating, know my separate rating in singles, doubles, and mixed.

I think separate ratings for singles vs doubles is a good thing.

About revealing exact DNTRP: Personally, I would like that. It would give me a more accurate way to gauge my progress (or most likely, lack thereof). However I do think it will lead to more and better sandbagging. I happen not to care that much - if people want to sandbag, that's their problem not mine. But most people on this board will care.

[ian2]

Originally Posted by OrangePower

I'm sure one could come up with another layer of rules to include/exclude scores under scenarios depending on DNTRP spread and so on. But then this is getting much more complicated and will in turn introduce more issues.

I think we agree on nearly everything... I'm not at all in favor of complicating the system any further. Ideally, I'd like every eligible match count. By "eligible" I mean a match where a) all players have DNTRP and b) the match is in a USTA league/tournament. I just don't trust the existing algorithm to handle well the additional level of disparity that mixed/combo introduces.

[OrangePower]

Originally Posted by ian2

I think we agree on nearly everything... I'm not at all in favor of complicating the system any further. Ideally, I'd like every eligible match count. By "eligible" I mean a match where a) all players have DNTRP and b) the match is in a USTA league/tournament. I just don't trust the existing algorithm to handle well the additional level of disparity that mixed/combo introduces.

Yeah, I agree that the rating algorithm could be better, although it kinda works ok. I'm not sure how the USTA decided on it and why. Part of the complexity (and resulting issues) is because the algorithm looks at scores rather than simple win/lose.

If I was in charge, I would investigate a simpler approach, based on the ELO algorithm applied to each set played. What this means is that each set won/lost would cause your rating to go up/down, with the amount of the adjustment depending on the difference between your DNTRP and the DNTRP of your opponent. The actual score within each set makes no difference using this approach. Players without a DNTRP would be assigned one based on their self-rating. Doubles results can be computed in exactly the same way, by using the combined DNTRPs of each team. Also, this algorithm is able to accommodate broad variation in the ratings of each person/team, so that every single sanctioned match can be considered.

The ELO algorithm is tried and trusted, and has been used successfully for calculating ratings in other sports/activities where there can be a wide range in the skill levels of players. Its most well-known application is for Chess ratings.

[Cindysphinx]

Oh, dear God. Let's not release exact ratings. I cannot imagine the backtalk I would get from my players.
"How come I don't get Superstar as a partner? My rating is closer to hers than anyone else on the team!"

"Why do I have to play Line 1? She has a higher rating by .02. Don't sacrifice me!"

It is bad enough that folks with a higher NTRP think it is some sort of validation, regardless if it was a fluke. Let's not make it worse.

[dizzlmcwizzl]

^^^ I agree with the sphinx that while I personally want to know see every ones exact ratings, if that were true I would spend many more hours on tennis link. I would spend hours comparing my ratings and results to my opponents and who they played etc. I would probably spending hours trying to duplicate the USTA algorithm just for fun. I do not need that level of information.

However, with the new sign in feature on Tennis Link I would not mind if I were able to know my exact rating and not see any others.

[cak]

A few years back NorCal tried adding in a new system where you could see your, and everyone else's DNTRP out to three places. The idea was you enter your daily practice matches to get more data points to make the ratings better reflect reality.

Other than the obvious "no one wants to enter scores everyday" and "some times I just go out for fun and the scores don't matter", the whole experiment was a huge disaster.

There was a ridiculous amount of whining, just as Cindy predicted. From the first day you could see ratings it was obvious the system was flawed. (When lining up people whose games you know, the ones that were very good were sometimes below those that weren't so good.)

And there was one guy entering scores for himself even though he was injured and hadn't played in months...

But most importantly, people would spend energy manipulating their scores by sandbagging matches or lining up matches so they wouldn't affect ratings. If you know you are close to getting DQ'd, and it would mean your team would drop out of playoff contention, there are quite a few people willing to drop a few games or a set for the team.

[ian2]

Originally Posted by cak

A few years back NorCal tried adding in a new system where you could see your, and everyone else's DNTRP out to three places. The idea was you enter your daily practice matches to get more data points to make the ratings better reflect reality.

Other than the obvious "no one wants to enter scores everyday" and "some times I just go out for fun and the scores don't matter", the whole experiment was a huge disaster.

There was a ridiculous amount of whining, just as Cindy predicted. From the first day you could see ratings it was obvious the system was flawed. (When lining up people whose games you know, the ones that were very good were sometimes below those that weren't so good.)

And there was one guy entering scores for himself even though he was injured and hadn't played in months...

But most importantly, people would spend energy manipulating their scores by sandbagging matches or lining up matches so they wouldn't affect ratings. If you know you are close to getting DQ'd, and it would mean your team would drop out of playoff contention, there are quite a few people willing to drop a few games or a set for the team.

Practice matches?! Self-entered scores?! No one here is advocating this. I find it hard to believe that NorCal would introduce such system... it's just nuts. Are you sure this was an official NorCal thing, and not someone's "simulation" website? Kind of like simulation stock trading, to explore "what if" scenarios?

[SweetH2O]

Originally Posted by ian2

Practice matches?! Self-entered scores?! No one here is advocating this. I find it hard to believe that NorCal would introduce such system... it's just nuts. Are you sure this was an official NorCal thing, and not someone's "simulation" website? Kind of like simulation stock trading, to explore "what if" scenarios?

Sounds a lot like what would happen if USTA went with a Tencap system, which it seemed like they were considering with the questionaire they sent out a few months back.