Three Simple Ways to Improve the ATP Ranking System

Rafael Nadal‘s two-year ranking system would favor a few veterans at the expense of everyone else.  My algorithm is too complex for players and fans to use on a weekly basis.  But there is always an undercurrent of dissatisfaction over the current system.

The rankings serve two main purposes, each of which we must keep in mind as we think through a better system:

  • Entertainment. The fans want to know who’s number one.  No system will ever be perfect, but if the ranking system told us that Nadal outranked Djokovic despite losing to him several times in a row, the system would lose credibility.
  • Tournament entry. Rankings determine who gets direct entry into tournaments.  A biased ranking system would keep stronger players out of tournaments while letting in lesser players.

A system that is good for one of these purposes is generally good for the other.  In an ideal world, the rankings would show us who is playing the best right now, carefully defining “right now” to avoid an unnecessary focus on current hot streaks.  Another way to look at is that the rankings should be as predictive as possible.  If underdogs are constantly winning, that doesn’t mean tennis is a sport full of triumphant underdogs, it means we’re ranking players incorrectly!

The current system isn’t that bad.  There are three main problems, however:

  1. Last week is equal to last year.  The winner in Miami this week will gain 1000 points.  Those 1000 points will be counted in his ranking next week, in six months, and in 51 weeks. In 53 weeks, though, he’ll have zero points.  If we’re trying to measure how good he is, a tournament 51 weeks ago isn’t nearly as informative as his tournament last week.  And if we insist on using his result from 51 weeks ago, why not his result from 53 weeks ago?
  2. Surfaces are interchangeable.  Milos Raonic won a slew of matches on indoor courts last spring, which earned him a seed at the French Open.  Now, I love Milos, but did he really deserve a seed at the French, despite virtually no professional experience on clay?  Performance on one surface translates to other surfaces to some extent, but (obviously!) all surfaces are not created equal.
  3. All opponents are equal.  In the Miami third round, Andy Roddick beat Roger Federer … then lost.  He’ll get 90 points. Kei Nishikori beat Lukas Rosol … then lost. These sorts of differences sometimes even out over time, but must we trust that they will?  Roddick’s achievement this week is much more impressive than Nishikori’s, and should be treated as such.

We can fix all of these problems with simple arithmetic, making tweaks to the system that any player or fan can understand.

In these solutions, the exact details don’t matter.  The most important thing is simply to acknowledge that not all matches are equal.

  1. Last week is worth more than last year.  In my system, last week is worth a little bit more than the week before, which is worth a little bit more than the week before that, and so on.  Here’s a simple way to incorporate that into the ATP system: After four months, tournaments are worth only 80% of their original points.  After eight months, tournaments are worth only 60% of their original points.  That way, the drop off is more gradual, and Indian Wells is worth more than, say, the 2011 Rome Masters.  If Nadal still wants two years, this can easily be extended to cover two years of results–after a year, 45%; after 16 months, 30%, after 20 months, 15%.  Now everybody’s happy!
  2. Separate surfaces, separate rankings.  There will always–and should always–be a single most important ranking list, encompassing all surfaces.  But for tournament entry, why not do better?  For example, create a clay list by doubling the point value of all clay tournaments and leaving the others alone.  David Ferrer and Carlos Berlocq will rise; John Isner and Kevin Anderson will fall.  Any tennis fan knows this happens, so tournaments should determine entry this way, as well.  After all, Wimbledon has long used this sort of approach for seeding, if not for direct entry.
  3. Bonus points for beating top players.  The WTA used to do this, and it’s the least straightforward of my suggestions.  It’s so important, though, that a little complexity is worth a lot.  Let’s say 100 points for a win over anyone in the top 3; 75 points for beating anyone ranked 4, 5, or 6; 50 points for a win over anyone else in the top 10, 30 points for beating anyone ranked 11-15, and 10 points for a win over anyone ranked 16-20.  Mega-upsets like those scored lately by Isner, Roddick, and Grigor Dimitrov tell us something important, and the rankings should listen.
This is all stuff you can do on a calculator–nothing is more complex than the rules governing protected rankings or zero-pointers.  Young players will see their rankings rise more quickly once they begin beating the top guys.   All players will get into tournaments (and earn seeds) on surfaces where they have had more success .  And the fans will have a more accurate ranking system both to rely upon and to fuel arguments about which players are really better.

2012 Miami Projections: Semifinals

It’s three of the big four … and Juan Monaco.  Pico has always been better on hard courts than he gets credit for, but after knocking out the former and current American #1s in successive rounds, he finds himself a little out of his depth.

Here are the odds for today’s semifinal matches:

Player                 F      W  
(1)Novak Djokovic  84.9%  50.5%  
(21)Juan Monaco    15.1%   3.2%  
(4)Andy Murray     49.3%  22.7%  
(2)Rafael Nadal    50.7%  23.6%

Also, here are my probabilities for every possible final matchup.  The number in each row is the percentage chance that the player on the left beats the player on the right.  In other words, the top row says: “If Djokovic faces Murray, he has a 59.9% chance of winning.”

Player             Opponent            P(W)  
(1)Novak Djokovic  (4)Andy Murray     59.9%  
(1)Novak Djokovic  (2)Rafael Nadal    59.3%  
(21)Juan Monaco    (4)Andy Murray     21.2%  
(21)Juan Monaco    (2)Rafael Nadal    20.8%  
(4)Andy Murray     (1)Novak Djokovic  40.1%  
(4)Andy Murray     (21)Juan Monaco    78.8%  
(2)Rafael Nadal    (1)Novak Djokovic  40.7%  
(2)Rafael Nadal    (21)Juan Monaco    79.2%

The Fatal Flaw of Nadal’s Two-Year Ranking System

Now that Rafael Nadal has resigned from the ATP player council–apparently because no one took his two-year ranking plan seriously–we’re likely to hear a bit more about this alternate approach.

Presumably, Nadal’s method would count the last 104 weeks (two years) of results instead of the last 52, as is currently the case.  As far as I know, he isn’t pushing for any other adjustments.  As long as that is the case, the rest of the council (and the ATP in general) is right to ignore Nadal’s plan: It would do significant damage to the sport, without much in the way of benefits.  It would drastically slow the rise of young players, but change little for guys at the top.

Ultimately, the question is over the purpose of the ranking system.  If it is to reward past performance, a two-year ranking system may be appropriate.  If it is to rank competitors by their current level of play, treating a tournament 22 months ago the same as last week’s tournament is flat-out wrong.

Consider what the present ranking system tells us.  By equally weighting tournaments over the last 52 weeks (with more points for more important events, of course), a player’s ranking is the average of how good he has been over the last 52 weeks–in other words, it’s a approximation of how good he was 26 weeks ago.  For most players, this is a decent estimate of how good they are right now.  If we go to a two-year system, the rankings would be an estimate of how good players were one full year ago.  Yikes.

The most obvious casualties of such a system are young players (or any players, really) on the way up.  Even with the current system, the rankings take some time to catch up with a rising star like Bernard Tomic or Milos Raonic.  When Raonic had his great run in early 2011, the rankings were still counting a bunch of challenger results from one year earlier.  In a two-year system, Raonic’s more recent results would count for even less.  It would take twice as long for such a player to establish himself.

The clear beneficiaries, of course, are the opposite type of competitor: established players who are declining or injured.  If a player is consistently good, it really doesn’t matter how the ranking system is calculated–just about any way you slice it, Djokovic, Nadal, Federer, and Murray would be the top four.  But the players who benefit are the ones who posted good results between 52 and 104 weeks ago, and haven’t done nearly as well since.  Right now, that means injured players like Robin Soderling, and declining players like Andy Roddick and Fernando Verdasco.

Should Roddick and Verdasco continue to be rewarded for their play in 2010?  To me, anyway, the answer is a clear “no.”  Even with Roddick’s sharp decline, he will probably still earn a seed for the French Open.  Does he deserve more than that?

But what about Soderling?  He hasn’t played since June, and his ranking has fallen to #30.  Unless he returns in the next three months, he’ll fall off the list altogether.  If there is a case for Nadal’s system, this is it.  But the ATP already has two methods in place to protect players like Soderling: protected rankings (PR) and wild cards.  Players injured for a certain length of time are able to use a PR (equal to their ranking when they last played) for entry to a set number of tournaments.  Until recently, Tommy Haas was still using a PR of 20.  Soderling would have a PR that would get him into enough tournaments to rebuild his ranking, assuming he comes back with any semblance of his previous form.

Of course, there’s also the wild card.  When Soderling returns, even if he is unranked, every 250- and 500-level tournament would hand him a wild card without a second thought.  This makes PRs even more valuable than the ATP intended them to be: Haas, for example, has been able to use his PR of 20 for so long because many tournaments gave him wild cards.  He could save the PR for when he needed it.

The only disadvantage to PRs and WCs is that these players aren’t seeded.  But really, after sitting out for a year, does a player deserve safe passage to the third round?  I find it hard to believe that they do.  And if this is really such an important issue, perhaps a player such as Soderling could be granted the lowest seed (e.g. 32, at Indian Wells, Miami, or a slam) two of the times he uses his protected ranking.

To recap: A simple two-year system would retard the rise of young players, forcing them to prove themselves for twice as long as is currently the case.  It wouldn’t affect consistently good players.  It would help players on the decline who probably don’t deserve help.  And top players returning from injury have little problem entering tournaments; Nadal’s approach would just get them seeds.

But Jeff, doesn’t your ranking system use two years of results?

Yes, I was getting to that.  It’s crucial to distinguish between using two years of results (acceptable) and weighting all results equally (unacceptable).

The biggest problem with the ATP ranking system as is–and it would be an even bigger problem with a two-year system like Nadal’s–is that it treats long-ago tournaments as equal to yesterday’s tournaments.  The winner of the 2012 Indian Wells event has 1000 points on his ranking.  The winner of the 2011 Miami even has 1000 points on his ranking.  The winner of the 2011 Indian Wells event has … zero points on his ranking.

How a player performed 18 months ago, or 20 months ago, has some predictive value.  But not nearly as much as the predictive value of their more recent performances.  In slight support of Nadal’s case, this is particularly true of players returning from injury.  My system never removed Juan Martin del Potro from the top 10 or so; using a one-year system, the ATP rankings saw him drop far out of the top 100.

If you are to use two years of results, it is absolutely imperative to differentiate between recent results and older results.  In fact, even a simple approach of this sort would improve the current 52-week system.  My algorithm weights results one year ago about half as heavily as last week’s, and two years ago roughly one-quarter as heavily.  The weighting is not simple, and thus would be inappropriate for the ATP system, which must be easily understood by both players and fans, but it points the way toward simpler solutions that might work.

That’s enough for today.  Check back tomorrow, when I’ll go into more depth about how the current ranking system can be improved.

2012 Miami Projections: Quarterfinals

For the second week in a row, Novak Djokovic gets the easy half, as someone cleared out his potential semifinal opponent.  In the other half, Andy Murray gets the easy quarter while Nadal has to get past Tsonga.  Here are my projections for these four matches and beyond:

Player                    SF      F      W  
(1)Novak Djokovic      76.5%  60.8%  38.5%  
(5)David Ferrer        23.5%  12.9%   4.5%  
(21)Juan Monaco        37.0%   7.5%   1.8%  
(8)Mardy Fish          63.0%  18.8%   6.6%  
(9)Janko Tipsarevic    27.1%   8.3%   2.5%  
(4)Andy Murray         72.9%  39.3%  20.4%  
(6)Jo-Wilfried Tsonga  38.7%  17.7%   7.4%  
(2)Rafael Nadal        61.3%  34.7%  18.3% 

2012 Miami Projections: Fourth Round

Men’s tennis may be predictable these days, but it’s not that predictable.  Andy Roddick beat Roger Federer last night, handing Federer his first loss in a best-of-three since August, sending home the hottest player in the game.  (It’s also Fed’s first lost to anyone outside of the top 20 in almost two years. The last one was his 2010 loss to Lleyton Hewitt in Halle.) Federer’s exit makes life easier for Novak Djokovic, who would’ve faced Fed in the semis, and does an even bigger favor for Mardy Fish, who would’ve played Roger in the quarters.

11 of the top 16 seeds are still alive, but things have definitely gotten more interesting.

Player                        QF     SF      F      W  
(1)Novak Djokovic          80.5%  56.5%  45.4%  30.0%  
(17)Richard Gasquet        19.5%   7.2%   3.6%   1.2%  
(11)Juan Martin Del Potro  62.6%  25.2%  17.1%   8.7%  
(5)David Ferrer            37.4%  11.1%   6.2%   2.4%  

(31)Andy Roddick           55.6%  25.6%   6.8%   2.2%  
(21)Juan Monaco            44.4%  18.0%   4.0%   1.1%  
(12)Nicolas Almagro        41.0%  21.0%   5.4%   1.6%  
(8)Mardy Fish              59.0%  35.5%  11.5%   4.5%  

Player                        QF     SF      F      W  
Grigor Dimitrov            35.3%   7.7%   1.8%   0.4%  
(9)Janko Tipsarevic        64.7%  21.5%   7.4%   2.5%  
(13)Gilles Simon           27.0%  14.9%   5.2%   1.7%  
(4)Andy Murray             73.0%  55.9%  32.5%  17.9%  

(6)Jo-Wilfried Tsonga      65.4%  29.2%  14.9%   6.7%  
(19)Florian Mayer          34.6%  10.5%   3.8%   1.2%  
(16)Kei Nishikori          31.4%  14.9%   6.5%   2.5%  
(2)Rafael Nadal            68.6%  45.4%  27.9%  15.5%

The Unaceables

Last night, Florian Mayer solved the John Isner serve, breaking the American three times en route to a straight-set victory.  Mayer is known as a tricky opponent, but not as a particularly good returner.  He had never played Isner before, though he beat Ivo Karlovic in Miami last year.

One element of his success is that he got his racquet on the Isner serve.  Over the last 52 weeks, Isner has amassed a 17.1% ace rate, meaning that about one in six of his serves are untouchable.  Last night, he barely managed 10%, as Mayer allowed him only six aces.

We might wonder: Is this is a skill of Mayer’s that we’ve failed to notice before?  At first glance, it doesn’t appear to be.  While Mayer often holds his opponents to low ace numbers, he’s had some horrible performances in that department, allowing Feliciano Lopez a 20.4% ace rate in Shanghai last year, Thomaz Bellucci 15.5% in Madrid on clay, and while playing injured, he ignominiously allowed Ivo Karlovic a 50% ace rate at last year’s Cincinnati Masters.

We can answer this question not just for Mayer, but for every regular on the ATP tour.  While some servers hit far more aces than others, ace rate is influenced by both the server and the returner.  Mayer himself is a good example.  In the last 52 weeks, he’s had eight matches in which at least one in ten serves went for an ace.  But in five other matches, he didn’t hit a single one!  Some of the variation is due to good and bad serving performances, but a substantial part can be explained by the man on the other side of the net.

As  it turns out, last night was an aberration for the German.  Mayer is below-average at ace prevention, allowing 8% more aces than an average player, ranking 80th among the 139 active players whose results I analyzed.

I looked at every 2011 and 2012 match, using only those matches in which both players racked up 10 matches in the last fifteen months.  After calculating each player’s ace rate, I generated an “expected” number of aces for each returner.  Simply tallying how many aces a player allowed isn’t good enough–this way, we adjust for the quality of the server.

Mayer, for instance, played 70 matches in that span against opponents who also played at least 10 matches.  (I excluded guys who played fewer than 10 because their ace rate in such a small number of matches may say more about their opponents than themselves.)  In his 4812 return points, he allowed 345 aces.  But based on the serving abilities of his opponents, he should have allowed only 321.  Those numbers will look a little better after last night, but not enough to move him up very much in the rankings.

By contrast, the best returners get their racquets on just about everything.  Atop the list is Gael Monfils, who allows barely half the aces that we would expect him to.  The top eight returners all reduce expected ace rates by at least a third.

In the table below, I’ve shown these stats for the ten players who appear to be the best at avoiding aces, along with 20 other players of interest.

Player                 Rank  Matches  vAce%  expAce%    Diff  
Gael Monfils              1       62   3.5%     6.8%    -48%  
Benoit Paire              2       23   3.8%     6.3%    -40%  
Andy Murray               3       81   4.4%     7.3%    -39%  
Stanislas Wawrinka        4       61   4.2%     7.0%    -39%  
Cedrik Marcel Stebe       5       12   3.2%     5.2%    -38%  
Viktor Troicki            6       70   4.3%     7.0%    -38%  
Gilles Simon              7       77   4.7%     7.3%    -36%  
David Ferrer              8       90   5.1%     7.8%    -35%  
Carlos Berlocq            9       53   4.7%     7.0%    -32%  
Mardy Fish               10       71   5.7%     8.3%    -31%  

Jo Wilfried Tsonga       14       89   5.7%     7.9%    -28%  
Roger Federer            20       92   6.0%     7.9%    -24%  
Novak Djokovic           22       89   6.4%     8.4%    -24%  
Kei Nishikori            32       63   5.8%     7.0%    -17%  
Rafael Nadal             34       91   7.4%     8.8%    -16%  
Nikolay Davydenko        38       60   5.8%     6.7%    -14%  
Sam Querrey              39       35   6.7%     7.8%    -14%  
Milos Raonic             40       60   6.7%     7.6%    -12%  
Kevin Anderson           53       74   7.5%     8.0%     -6%  
John Isner               59       68   7.6%     7.8%     -2%  

Radek Stepanek           73       62   8.6%     8.0%      6%  
Lukasz Kubot             74       44   8.5%     8.0%      7%  
Ivo Karlovic             78       45   7.9%     7.3%      7%  
Juan Martin Del Potro    81       84   8.8%     8.1%      9%  
Tomas Berdych            91       87   8.5%     7.6%     12%  
David Nalbandian        102       43   9.4%     7.9%     20%  
Arnaud Clement          120       17   9.3%     7.2%     29%  
Andy Roddick            130       55  11.8%     8.3%     42%  
Bernard Tomic           135       38  12.8%     8.5%     50%  
Olivier Rochus          139       36  14.7%     7.2%    103%

Before we go anointing Monfils and Benoit Paire the greatest returners in the game, it’s important to remember the serious limitations of the ace stat.  Much more important is getting the return in play.  But except for Grand Slam matches, we don’t have those numbers. In the meantime, we can use ace rate and return points won as proxies for return skills.

2012 Miami Projections: 3rd Round

The 2nd round went almost precisely according to script, leaving us with some high-profile matchups for a mere third round.  The closest third-rounders on paper (my paper, anyway) are Almagro/Verdasco and Tipsarevic/Dolgopolov.  While Federer and (particularly) Murray have big-serving challenges this round, Djokovic and Nadal are set to coast.  The only question is: How many games will Novak allow Troicki to win this time?

Player                       R16     QF     SF        W  
(1)Novak Djokovic          85.0%  70.8%  52.1%    25.2%  
(27)Viktor Troicki         15.0%   7.3%   2.5%     0.2%  
(17)Richard Gasquet        79.2%  20.0%   8.4%     1.1%  
Albert Ramos               20.8%   1.9%   0.3%     0.0%  
(11)Juan Martin Del Potro  67.7%  45.2%  20.0%     6.0%  
(23)Marin Cilic            32.3%  15.9%   4.7%     0.6%  
(30)Julien Benneteau       36.1%  11.0%   2.6%     0.2%  
(5)David Ferrer            63.9%  27.9%   9.4%     1.6%  

Player                       R16     QF     SF        W  
(3)Roger Federer           75.0%  52.3%  38.6%    12.3%  
(31)Andy Roddick           25.0%  11.1%   5.4%     0.5%  
(21)Juan Monaco            33.2%   9.0%   3.9%     0.3%  
(14)Gael Monfils           66.8%  27.7%  16.8%     2.9%  
(12)Nicolas Almagro        50.8%  23.0%   7.5%     0.7%  
(20)Fernando Verdasco      49.2%  21.8%   7.0%     0.6%  
(28)Kevin Anderson         38.7%  18.7%   5.8%     0.5%  
(8)Mardy Fish              61.3%  36.5%  15.0%     2.2%  

Player                       R16     QF     SF        W  
(7)Tomas Berdych           74.7%  47.0%  23.3%     4.7%  
Grigor Dimitrov            25.3%   9.0%   2.2%     0.1%  
(18)Alexandr Dolgopolov    48.8%  21.1%   7.8%     0.8%  
(9)Janko Tipsarevic        51.2%  23.0%   8.7%     1.0%  
(13)Gilles Simon           57.8%  19.2%   8.8%     1.0%  
(22)Jurgen Melzer          42.2%  11.3%   4.3%     0.3%  
(26)Milos Raonic           27.1%  14.5%   6.8%     0.8%  
(4)Andy Murray             72.9%  54.9%  38.1%    12.1%  

Player                       R16     QF     SF        W  
(6)Jo-Wilfried Tsonga      68.1%  40.5%  19.5%     4.4%  
(32)Philipp Kohlschreiber  31.9%  13.1%   4.2%     0.4%  
(19)Florian Mayer          39.8%  16.2%   5.5%     0.6%  
(10)John Isner             60.2%  30.2%  13.0%     2.4%  
(16)Kei Nishikori          77.3%  29.0%  14.1%     2.3%  
Lukas Rosol                22.7%   3.9%   0.9%     0.0%  
(25)Radek Stepanek         16.0%   5.7%   1.6%     0.1%  
(2)Rafael Nadal            84.0%  61.5%  41.2%    13.9%