The Most (and Least) Consistent Players on the ATP Tour

“Consistency” is one of the many terms that commentators frequently use but rarely define.  It’s often misused, too: we say we want a player to be more consistent, when we really just want him to stop playing badly.

To me, consistency for a tennis player is similar to the notion of “playing up to his ranking.”  In other words, if a player is consistent, he usually beats players ranked lower, and he usually loses to players ranked higher.  No player is perfect in this regard, but clearly, some are much more reliable than others.

A recent poster boy for inconsistency is Ernests Gulbis.  At Roland Garros, he lost to Blaz Kavcic, ranked 82nd in the world.  That was on clay, a surface on which Gulbis had posted some excellent results the previous year.  Two months later, in Los Angeles, he beat Juan Martin del Potro, someone he shouldn’t have even challenged on a hard court.

Quantifying consistency

With my player rankings and match prediction system, I’m able to assign a win probability to each player for every match.  For instance, when Ivan Dodig beat Nadal last week, I had given him a 14.4% chance of doing so.  As you might imagine, that’s a major upset–as I wrote the next day, it was the 10th-biggest upset of the season.

In these terms, an ideally consistent player will never be on either end of an upset.  If he is the favorite, he wins; if he is the underdog, he loses.  In practice, no tour-level player accomplishes this, though over the last two years, Florent Serra and Eduardo Schwank have come very close.

I’ve come up with a metric to measure consistency.  This is how it works:

  • Gather a list of all ATP-level matches for the desired time period.  (Today, I’m using everything from January 2010 through Montreal last week.)
  • Eliminate matches that ended in retirement or walkover, as well as those where we don’t have enough information to make an educated prediction.  (e.g. the first few comeback matches of Tommy Haas, or one with a wildcard playing his first professional match.)
  • For each player, count how many matches he played.
  • For each player, find the matches where he was the favorite and lost, or was the underdog and won.
  • For each of those matches, take the probability than the eventual winner would win (e.g. 18% — always under 50%), multiply by 100 (e.g. 18, not 18%),  subtract it from 50 (e.g. 50 – 18 = 32), and square the result (e.g. 32*32 = 1024).
  • Sum all of the squares, then divide by the number of total matches–not just the ones where the favorite lost.

Whew!  In something more like layman’s terms, we’re taking all the upsets a player was involved in, coming up with a number to represent how big (or surprising) the upset was, then averaging the results.

Using this method, we give big upsets considerably more weight than mini-upsets.  If a player had a 45% chance of winning a match and ends up winning, it barely counts as an upset–and this system treats it accordingly.  By dividing by the total number of matches, we give consistency credit to players who win the matches they’re “supposed to” win, and lose those they are supposed to lose.

Most importantly, the numbers this algorithm spits out are completely believable, matching up well with the conventional wisdom of which players are consistent and inconsistent.

The consistency of the top ten

The most consistent player on the tour, since the beginning of 2010, has been … Florent Serra.  Amazingly, Igor Kunitsyn comes in second.  But I doubt many of you care much about the consistency of guys like that.

Let’s start with the current top 10, ranked from most to least consistent:

Player              Upsets  Matches    Up%  UpsetScore  
David Ferrer            25      119  21.0%          55  
Rafael Nadal            16      131  12.2%          68  
Novak Djokovic          18      119  15.1%          69  
Roger Federer           21      123  17.1%          69  
Jo-Wilfried Tsonga      20       80  25.0%          75  
Mardy Fish              19       77  24.7%          82  
Tomas Berdych           32      113  28.3%         106  
Gael Monfils            24       89  27.0%         107  
Robin Soderling         23      115  20.0%         130  
Andy Murray             24       97  24.7%         151

The relevant column is the rightmost, “UpsetScore,” which is the result of the algorithm described above.  Ferrer has been part of more upsets than any of the top three (“Up%”), but his upsets are more minor.  Except for losses to Ivo Karlovic and Jarkko Nieminen early in the year on hard courts, Ferrer has not lost a match he had a 60% or better chance of winning.

The two ends of this list certainly line up with what I would have expected: Ferrer and Nadal are rock-solid (last week’s loss to Dodig notwithstanding), while Soderling and Murray both can be picked off by anybody, and frequently threaten higher-ranked players.

Right now, you may be tempted to put Djokovic higher on the list–after all, he’s ranked #1 and he’s beating everybody.  However, in the slightly longer term of 20 months, his movement around the top three has included some unexpected results, like losing to Ljubicic at Indian Wells last year, and victories over Federer and Nadal before his ranking suggested he would do so.

Tour wild cards

Outside of the top 10, there are a handful of players who are almost impossible to predict.  Some names that come to mind are Marcos Baghdatis, Ernests Gulbis, and Nikolay Davydenko, men who can take out one of the top three on a good day (well, maybe not Gulbis), but can lose to a qualifier on the next.

Player                 Upsets  Matches  Up%  UpsetScore  
Nikolay Davydenko          32       73  44%         273  
Marin Cilic                27       91  30%         180  
Marcos Baghdatis           38       89  43%         177  
Olivier Rochus             20       52  38%         164  
Milos Raonic               16       41  39%         164  
Juan Martin del Potro      11       48  23%         154  
Andy Murray                24       97  25%         151  
Jurgen Melzer              28       96  29%         150  
Fernando Verdasco          40      104  38%         150  
Ivan Ljubicic              27       69  39%         149  
Florian Mayer              30       72  42%         147  
Samuel Querrey             26       66  39%         146  
Andrei Goloubev            24       55  44%         143  
Ernests Gulbis             22       69  32%         140  
Jeremy Chardy              31       65  48%         133  
Juan Monaco                26       73  36%         131  
Robin Soderling            23      115  20%         130  
Michael Llodra             26       67  39%         130  
Rainer Schuettler          15       42  36%         119  
Mikhail Youzhny            25       78  32%         116

The “upset score” number tells the story for Davydenko.  The man who beat Nadal at the beginning of the year and threatened Djokovic last week recently suffered defeat at the hands of Cedrik-Marcel Stebe (twice!) and Antonio Veic.

While no one is in Davydenko’s league, names like Cilic, Baghdatis, Murray, and Verdasco seem appropriate.  Verdasco, along with Melzer and Milos Raonic suggest a flaw in this approach: the algorithm reads very fast improvement or decline as inconsistency, which isn’t quite right.  Yes, Raonic has shocked the tennis world repeatedly this season, but he hasn’t mixed in too many disastrous losses alongside the surprise upsets.  I tinkered with ways to include that in the model, but nothing worked very well.

A couple more interesting notes from the “most inconsistent” players are found in the upset percentage column.  Guys like Davydenko, Baghdatis, Mayer, Goloubev, and Chardy are involved in upsets nearly half the time.  Chardy is highest in that category.  In fact, if I expanded the study to challenger events, he might rocket to the top of this list, as he plays quite a few, and often manages to lose against players outside the top 100.

The consistent ones

The flip side is considerably less star-studded.  In the 20 most-consistent players of the last 19-20 months, Ferrer is the only top-10 guy present, though #11 Nicolas Almagro is there as well.

Here’s my seat-of-the-pants theory.  In this sense, “consistent” isn’t good.  Yes, “consistent” sounds good, especially when “inconsistent” means Davydenko losing to Antonio Veic or Mayer falling to Federico del Bonis.  But inconsistent means Davy beating Federer and Mayer beating Soderling.  So, the players who show up on as “most consistent” are in fact consistent, but they are also mediocre.  Their consistency (perhaps a mental advantage) has helped them move up from the top 200 to the top 50 or 100, but that’s all they can do.

Ferrer and Almagro are good examples of this, actually.  Neither has the weaponry that makes commentators say, “This guy could be number one!”  But they’ve earned their rankings by regularly reaching the quarters and semis of tournaments, not suffering the boneheaded losses that afflict the likes of Cilic and Baghdatis.

All that said, here’s the list:

Player            Upsets  Matches  Up%  UpsetScore  
Florent Serra         11       56  20%          23  
Igor Kunitsyn         14       40  35%          33  
Ilia Marchenko        14       46  30%          40  
Potito Starace        28       81  35%          46  
Victor Hanescu        26       77  34%          50  
Tobias Kamke          12       41  29%          52  
Andreas Seppi         24       81  30%          53  
Julien Benneteau      23       59  39%          53  
Viktor Troicki        25      101  25%          54  
David Ferrer          25      119  21%          55  
Fabio Fognini         18       71  25%          55  
Pere Riba             13       41  32%          56  
Lukas Lacko           14       44  32%          57  
Igor Andreev          17       62  27%          58  
Lukasz Kubot          26       63  41%          59  
Nicolas Almagro       22      112  20%          59  
Frederico Gil         15       40  38%          60  
Denis Istomin         25       76  33%          65  
Jarkko Nieminen       25       74  34%          66  
John Isner            21       82  26%          67

These lists hardly represent the final word on who is or is not consistent–for one thing, I haven’t said anything about consistency within matches, which may be a completely separate issue.  But this approach does, I think, provide some insight into who is more likely to be part of an upset, and suggests that consistency might not be such a good thing after all.

One thought on “The Most (and Least) Consistent Players on the ATP Tour”

Comments are closed.