The Power of One Point Per Thousand

Last week, I offered a method to rank smash-hitting skill. I measured the results in “points per 100”–the number of points a player could expect to gain or lose, relative to tour average, thanks to their ability hitting that one shot. The resulting figures were quite small: My calculations showed that Jo-Wilfried Tsonga has the game’s best smash, a shot worth 0.17 points per 100 above average, and 0.27 points per 100 above the weakest smash-hitting player I found, Pablo Cuevas.

That gap between best and worst of 0.27 per 100 gives us a rough maximum of how much difference a good or bad smash can make in a player’s game. The rate is roughly equivalent to one point out of 370. It sounds tiny, and since most players are closer to the average than they are to either of those extremes, the typical smash effect is even smaller still.

However, it’s difficult to have any intuitive sense of how much one point is worth. In any given match, a single point, or even five points, isn’t going to make the difference. On the other hand, plenty of matches are so close that one or two points would flip the result. If an average player could train really hard in the offseason and develop a smash just as good as Tsonga’s, what would that extra 0.17 points per 100 mean for him in the win column? What about in the rankings?

This is a relatively straightforward question to answer once we’ve posed it. Over the course of a season, the best players win more points than their peers–obviously. Yet the margin isn’t that great. In 2017, no man won points at a higher clip than Rafael Nadal, who came out on top 55.7% of the time. That’s less than seven percentage points higher than the worst player in the top 50, Paolo Lorenzi, who won 49.1% of points. Nearly half of top 50 players–22 of them–won between 49.0% and 51.0% of total points, and another 15% fell between 51.0% and 52.0%.

Fixing total points won

These numbers are slightly misleading, though only slightly. The total points won stat (TPW) tends to cluster very close to the 50% mark because competitors face what, in other sports, we would call unbalanced schedules. If you win, you usually have to play someone better in the next round; win again, and an even more superior opponent awaits. This means that the 6.6% gap between Nadal and Lorenzi is a bit wider than it sounds: Had the Italian faced the same set of opponents that Rafa did, he wouldn’t have managed to win 49.1% of points.

That problem, however, is possible to resolve. Earlier this year I shared an algorithm that analyzed return points won by controlling for opponent, by comparing how each pair of players fared in equivalent matchups. (That analysis hinted at the second-half breakthrough of return wizard Diego Schwartzman.) While we don’t know exactly what would happen if Lorenzi played Nadal’s exact schedule, we can use this common-opponent approach to approximate it. When we do so, we find that the 1st-to-50th, Nadal-to-Lorenzi spread is almost 10 percentage points; setting Rafa’s rate at a constant 55.7%, Lorenzi’s works out a less neutral-sounding 46.2%. Many players remain packed in the 49%-to-51% range, but the overall spread is wider, because we control for tennis’s natural tendency to cancel out player’s wins with subsequent losses.

Even when we widen the pool of players to 71–everyone who played at least 35 tour-level matches this season–the ten-percentage-point spread remains. Lorenzi remains close to the bottom, a few places above Mikhail Youzhny, whose competition-adjusted rate of points won is 45.7% ranks last, exactly ten points below Rafa.

Think about what that means: In a typical ATP match, for every hundred points played, only ten are really up for grabs. That isn’t literally true, of course: There are plenty of matches in which one player wins 60% or more of total points. But on average, you can expect even the weakest tour regular to win 45 out of 100 points. In team sports analytics, this is what we might call “replacement level”–the skill level of a freely available minor leaguer or bench player. I don’t like importing the concept of replacement level for tennis, because in an individual sport you’re never really replacing one player with another. But at the most general level, it’s a useful way of thinking about this subject–just as even a minor league batter could hit .230 in the major leagues (as opposed to .000), so a fringey ATP player will win 45% of points, not 0%.

Points to wins

In team sports analytics, it’s common to say that some number of runs, or goals, or points is equal to one win. Thinking in terms of wins is a good way to value players: If you can say that upgrading your goalkeeper is worth two wins over your current option, it makes very clear what he brings to the table. Again, the metaphor is a bit strained when we apply it to tennis, but we can start thinking about things in the same way.

Another oddity in tennis is that players not only face very unequal competition, they also play widely different numbers of matches. The year-end top 50 contested anywhere from 35 matches up to more than 80; part of the variation is due to injury, but much is structural: The more matches you win, the more you play. Rafa managed his schedule by entering only a handful of optional events, yet only David Goffin played more matches. So we have another quirk to handle: In this case, let’s adopt the fiction that a tennis season is exactly 50 matches long. Rafa’s actual record was 67-11; scaled to a 50-match season, that’s roughly 43-7.

Finally, we can look at the relationship between points and wins. Points, here, means the rate of total points won adjusted for competition. And wins is the number of victories in our hypothetical 50-match season. The relationship between points and wins is quite strong (r^2 = 0.75), though of course not exact. Roger Federer won matches at a higher rate than Nadal did, but by competition-adjusted total points won, Rafa trounced him, 55.7% to 53.5%. And as we’ve seen, Lorenzi is close to the bottom of our 71-player sample, despite hanging on to a ranking in the mid-40s. Luck, clutch play, and a host of other factors make the points-to-wins relationship imperfect, but it is nonetheless a healthy one.

It doesn’t take many points to boost one’s win total. An increase of only 0.367 points per 100 translates into one more win in a 50-match season. The average player contests 8,000 points per season, so we’re talking about only 29 more points per year. This puts my smash-skill conclusions in a new light: The spread between the best and the worst of 0.27 points per 100 seemed tiny, but now we see it’s worth almost a full win over the course of a 50-match season.

Wins to ranking places

Unless you’re nearing a round number and have a hankering for cake, even wins aren’t the currency that really matters in tennis. What counts is position on the ranking table. The relationship between wins and ranking position is another strong but imperfect one (r^2 = 0.63).

As we’ve seen, the middle of the ATP pack is tightly grouped together in total points won, with so many players hovering around the 50% mark, even when adjusted for competition. There’s not much to distinguish between these men in the win column, either: On average, an increase of 0.26 wins per 50 matches translates into a one-spot jump on the ranking computer. Put another way: If you win one more match, your ranking will improve by four places. Again, these are not iron laws–in reality, it depends when and where that extra win occurs, and the corresponding ranking improvement could be anywhere from zero spots to 30. Still, knowing the typical result allows us to understand better the impact of each marginal win and, by extention, the value of winning a few more points.

One point per thousand

Combine these two relationships, and we get a new, conveniently round-numbered rule of thumb. If an increase in one ranking place requires 0.26 additional wins per 50 matches, and one additional win requires 0.367 extra points per 100, a little tapping at the calculator demonstrates that one ranking place is equal to about 0.095 points per 100. Round up a bit to 0.1 per 100, and we’re looking at one point per thousand.

One extra point per thousand is a miniscule amount, the sort of difference we could never dream of spotting with the naked eye. Players regularly win entire tournaments without contesting so many points; even for Goffin, who served or returned more than 12,000 times this year, we’re talking about a dozen points. Yet think back to all of those players clustered between 49% and 52% of total points won; even when adjusted for competition, three men ended the 2017 season tied at exactly 50.4%, with less than one point per thousand separating the three of them.

The one part of the ranking table where one point per thousand is no more than a rounding error is the very top. Usually one player separates himself from the pack, and the top few distance themselves from the rest. This year is no different: The competition-adjusted gap between Nadal and Federer is a whopping 2.2% (22 points per thousand), while the next 2.2% takes us all the way from Fed through the entire top 10. The 2.2% after that, extending from 51.1% to 48.9%, covers another 20 players: spaced, on average, one point per thousand apart. For a player seeking to improve from 30th to 20th, the path is largely linear; from 5th to 3rd it is much less predictable–and probably steeper.

If this all sounds unnecessarily abstruse, I can only mention once again the example of my smash-skill findings. Now we know that the range of overhead-hitting ability among the game’s regulars is worth close to three places in the rankings. Imagine a similar type of conclusion for forehands, backhands, net approaches… it’s exciting stuff. While plenty of work lies ahead, this framework allows us to measure the impact of individual shots–perhaps even tactics–and translate that impact into ranking places, the ultimate currency of tennis.

Measuring the Best Smashes in Tennis

How can we identify the best shots in tennis? At first glance, it seems like a simple problem. Thanks to the shot-by-shot data collected for over 3,500 matches by the Match Charting Project, we can look at every instance of the shot in question and see what happened. If a player hits a lot of winners, or wins most of the ensuing points, he or she is probably pretty good at that shot. Lots of unforced errors would lead us to conclude the opposite.

A friend recently posed a more specific question: Who has the best smash in the men’s game? Compared to other shots such as, say, slice backhands, smashes should be pretty easy to evaluate. A large percentage of them end the point–in the contemporary men’s game (I discuss the women’s game later on), 69% are winners or induce forced errors–which reduces the problem to a straightforward one.

The simplest algorithm to answer my friend’s question is to determine how often each player ends the point in his favor when hitting a smash–that is, with a winner or by inducing a forced error. Call the resulting ratio “W/SM.” The Match Charting Project (MCP) dataset has at least 10 tour-level matches for 80 different men, and the W/SM ratio for those players ranges from 84% (Jeremy Chardy) all the way down to 30% (Paolo Lorenzi). Both of those extremes are represented by players with relatively small samples; if we limit our scope to men with at least 90 recorded smashes, the range isn’t quite as wide. The best of the bunch is Jo-Wilfried Tsonga, at 79%, and the “worst” is Ivan Lendl, at 57%. That isn’t quite fair to Lendl, since smash success rates have improved quite a bit over the years, and Lendl’s rate is only a couple percentage points below the average for the 1980s. Among active players with at least 90 smashes in the books, Stan Wawrinka brings up the rear, with a W/SM of 65%.

We can look at the longer-term effects of a player’s smashes without adding much complexity. It’s ideal to end the point with a smash, but most players would settle for winning the point. When hitting a smash, ATPers these days end up winning the point 81% of the time, ranging from 97% (Chardy again) down to 45% (Lorenzi again). Once again, Tsonga leads the pack of the bigger-sample-size players, winning the point 90% of the time after hitting a smash, and among active players, Wawrinka is still at the bottom of that subset, at 77%.

Here is a list of all players with at least 90 smashes in the MCP dataset, with their winners (and induced forced errors) per smash (W/SM), errors per smash (E/SM), and points won per smash (PTS/SM):

PLAYER              W/SM  E/SM  PTS/SM  
Jo-Wilfried Tsonga   78%    6%     90%  
Tomas Berdych        76%    6%     88%  
Pete Sampras         75%    7%     86%  
Roger Federer        73%    7%     86%  
Rafael Nadal         69%    7%     84%  
Milos Raonic         73%    9%     82%  
Andy Murray          67%    6%     82%  
Kei Nishikori        68%   11%     81%  
David Ferrer         71%    9%     81%  
Andre Agassi         67%    8%     80%  
Novak Djokovic       66%    9%     80%  
Stefan Edberg        62%   12%     78%  
Stan Wawrinka        65%   10%     77%  
Ivan Lendl           57%   13%     71%

These numbers give us a pretty good idea of who you should back if the ATP ever hosts the smash-hitting equivalent of baseball’s Home Run Derby. Best of all, it doesn’t commit any egregious offenses against common sense: We’d expect to see Tsonga and Roger Federer near the top, and we’d know something was wrong if Novak Djokovic were too far from the bottom.

Smash opportunities

Still, we need to do better. Almost every shot made in a tennis match represents a decision made by the player hitting it: topspin or slice? backhand or run-around forehand? approach or stay back? Many smashes are obvious choices, but a large number are not. Different players make different choices, and to evaluate any particular shot, we need to subtly reframe the question. Instead of vaguely asking for “the best,” we’d be better served looking for the player who gets the most value out of his smash. While the two questions are similar, they are not the same.

Let’s expand our view to what we might call “smash opportunities.” Once again, smashes make our task relatively straightforward: We can define a smash opportunity simply as a lob hit by the opponent.* In the contemporary ATP, roughly 72% of lobs result in smashes–the rest either go for winners or are handled with a different shot. Different players have very different strategies: Federer, Pete Sampras, and Milos Raonic all hit smashes in more than 84% of opportunities, while a few other men come in under 50%. Nick Kyrgios, for instance, tried a smash in only 20 of 49 (41%) of recorded opportunities. Of those players with more available data, Juan Martin Del Potro elected to go for the overhead in 61 of 114 (54%) of chances, and Andy Murray in 271 of 433 (62.6%).

* Using an imperfect dataset, it’s a bit more complicated; sometimes the shots that precede smashes are coded as topspin or slice groundstrokes. I’ve counted those as smash opportunities as well.

Not all lobs are created equal, of course. With a large number of points, we would expect them to even out, but even then, a player’s overall style may effect the smash opportunities he sees. That’s a more difficult issue for another day; for now, it’s easiest to assume that each player’s mix of smash opportunities are roughly equal, though we’ll keep in mind the likelihood that we’ve swept some complexity under the rug.

With such a wide range of smashes per smash opportunities (SM/SMO), it’s clear that some players’ average smashes are more difficult than others. Federer hits about half again as many smashes per opportunity as del Potro does, suggesting that Fed’s attempts are more difficult than Delpo’s; on those more difficult attempts, Delpo is choosing a different shot. The Argentine is very effective when he opts for the smash, winning 84% of those points, but it seems likely that his rate would not be so high if he hit smashes as frequently as Federer does.

This leads us to a slightly different question: Which players are most effective when dealing with smash opportunities? The smash itself doesn’t necessarily matter–if a player is equally effective with, say, swinging volleys, the lack of a smash would be irrelevant. The smash is simply an effective tool that most players employ to deal with these situations.

Smash opportunities don’t offer the same level of guarantee that smashes themselves do: In the ATP these days, players win 72% of points after being handed a smash opportunity, and 56% of the shots they hit result in winners or induced forced errors. Looking at these situations takes us a bit off-track, but it also allows us to study a broader question with more impact on the game as a whole, because smash opportunities represent a larger number of shots than smashes themselves do.

Here is a list of all the players with at least 99 smash opportunities in the MCP dataset, along with the rate at which they hit smashes (SM/SMO), the rate at which they hit winners or induced forced errors in response to smash opportunites (W/SMO), hit errors in those situations (E/SMO), and won the points when given lobs (PTW/SMO). Like the list above, players are ranked by the rightmost column, points won.

PLAYER              SM/SMO  W/SMO  E/SMO  PTW/SMO  
Jo-Wilfried Tsonga     80%    68%    13%      80%  
Roger Federer          84%    66%    13%      78%  
Pete Sampras           86%    68%    15%      78%  
Tomas Berdych          75%    66%    16%      76%  
Milos Raonic           85%    67%    14%      76%  
Novak Djokovic         81%    60%    13%      75%  
Kevin Anderson         66%    57%    12%      74%  
Rafael Nadal           74%    57%    16%      73%  
Andre Agassi           77%    62%    17%      73%  
Boris Becker           85%    59%    18%      72%  
Stan Wawrinka          79%    58%    15%      72%  
Kei Nishikori          72%    57%    17%      70%  
Andy Murray            63%    52%    15%      70%  
Dominic Thiem          66%    52%    11%      70%  
David Ferrer           71%    57%    17%      69%  
Pablo Cuevas           73%    54%    14%      67%  
Stefan Edberg          81%    52%    23%      65%  
Bjorn Borg             81%    41%    20%      63%  
JM del Potro           54%    48%    19%      60%  
Ivan Lendl             74%    45%    28%      59%  
John McEnroe           74%    43%    24%      56%

The order of this list has much in common with the previous one, with names like Federer, Sampras, and Tsonga at the top. Yet there are key differences: Djokovic and Wawrinka are particularly effective when they respond to a lob with something other than an overhead, while del Potro is the opposite, landing near the bottom of this ranking despite being quite effective with the smash itself.

The rate at which a player converts opportunities to smashes has some impact on his overall success rate on smash opportunities, but the relationship isn’t that strong (r^2 = 0.18). Other options, such as swinging volleys or mid-court forehands, also give players a good chance of winning the point.

Smash value

Let’s get back to my revised question: Who gets the most value out of his smash? A good answer needs to combine how well he hits it with how often he hits it. Once we can quantify that, we’ll be able to see just how much a good or bad smash can impact a player’s bottom line, measured in overall points won, and how much a great smash differs from an abysmal one.

As noted above, the average current-day ATPer wins the point 81% of the time that he hits a smash. Let’s reframe that in terms of the probability of winning a point: When a lob is flying through the air and a player readies his racket to hit an overhead, his chance of winning the point is 81%–most of the hard work is already done, having generated such a favorable situation. If our player ends up winning the point, the smash improved his odds by 0.19 points (from 0.81 to 1.0), and if he ends up losing the point, the smash hurt his odds by 0.81 (from 0.81 to 0.0). A player who hits five successful smashes in a row has a smash worth about one total point: 5 multiplied by 0.19 equals 0.95.

We can use this simple formula to estimate how much each player’s smash is worth, denominated in points. We’ll call that Point Probability Added (PPA). Finally, we need to take into account how often the player hits his smash. To do so, we’ll simply divide PPA by total number of points played, then multiply by 100 to make the results more readable. The metric, then, is PPA per 100 points, reflecting the impact of the smash in a typical short match. Most players have similar numbers of smash opportunities, but as we’ve seen, some choose to hit far more overheads than others. When we divide by points, we give more credit to players who hit their smashes more often.

The overall impact of the smash turns out to be quite small. Here are the 1990s-and-later players with at least 99 smash opportunities in the dataset along with their smash PPA per 100 points:

PLAYER                 SM PPA/100  
Jo-Wilfried Tsonga           0.17  
Pete Sampras                 0.11  
Tomas Berdych                0.11  
Roger Federer                0.10  
Rafael Nadal                 0.05  
Milos Raonic                 0.04  
Juan Martin del Potro        0.02  
Andy Murray                  0.01  
Kevin Anderson               0.01  
Kei Nishikori                0.00  
David Ferrer                 0.00  
Andre Agassi                 0.00  
Novak Djokovic              -0.02  
Stan Wawrinka               -0.07  
Dominic Thiem               -0.07  
Pablo Cuevas                -0.10

Tsonga reigns supreme, from the most basic measurement to the most complex. His 0.17 smash PPA per 100 points means that the quality of his overhead earns him about one extra point (compared to an average ATPer) every 600 points. That doesn’t sound like much, and rightfully so: He hits fewer than one smash per 50 points, and as good as Tsonga is, the average player has a very serviceable smash as well.

The list gives us an idea of the overall range of smash-hitting ability, as well. Among active players, the laggard in this group is Pablo Cuevas, at -0.1 points per 100, meaning that his subpar smash costs him one point out of every thousand he plays. It’s possible to be worse–in Lorenzi’s small sample, his rate is -0.65–but if we limit our scope to these well-studied players, the difference between the high and low extremes is barely 0.25 points per 100, or one point out of every 400.

I’ve excluded several players from earlier generations from this list; as mentioned earlier, the average smash success rate in those days was lower, so measuring legends like McEnroe and Borg using a 2010s-based point probability formula is flat-out wrong. That said, we’re on safe ground with Sampras and Agassi; the rate at which players convert smashes into points won has remained fairly steady since the early 1990s.

Lob-responding value

We’ve seen the potential impact of smash skill; let’s widen our scope again and look at the potential impact of smash opportunity skill. When a player is faced with a lob, but before he decides what shot to hit, his chance of winning the point is about 72%. Thus, hitting a shot that results in winning the point is worth 0.28 points of point probability added, while a choice that ends up losing the point translates to -0.72.

There are more smash opportunities than smashes, and more room to improve on the average (72% instead of 81%), so we would expect to see a bigger range of PPA per 100 points. Put another way, we would expect that lob-responding skill, which includes smashes, is more important than smash-specific skill.

It’s a modest difference, but it does look like lob-responding skill has a bigger range than smash skill. Here is the same group of players, still showing their PPA/100 for smashes (SM PPA/100), now also including their PPA/100 for smash opportunities (SMO PPA/100):

PLAYER                 SM PPA/100  SMO PPA/100  
Jo-Wilfried Tsonga           0.17         0.18  
Roger Federer                0.10         0.16  
Pete Sampras                 0.11         0.16  
Milos Raonic                 0.04         0.12  
Tomas Berdych                0.11         0.09  
Kevin Anderson               0.01         0.08  
Novak Djokovic              -0.02         0.07  
Rafael Nadal                 0.05         0.03  
Andre Agassi                 0.00         0.01  
Stan Wawrinka               -0.07         0.00  
Kei Nishikori                0.00        -0.03  
Andy Murray                  0.01        -0.03  
Dominic Thiem               -0.07        -0.05  
David Ferrer                 0.00        -0.06  
Pablo Cuevas                -0.10        -0.12  
Juan Martin del Potro        0.02        -0.19

Djokovic and Delpo draw our attention again as the players whose smash skills do not accurately represent their smash opportunity skills. Djokovic is slightly below average with smashes, but a few notches above the norm on opportunities; Delpo is a tick above average when he hits smashes, but dreadful when dealing with lobs in general.

As it turns out, we can measure the best smashes in tennis, both to compare players and to get a general sense of the shot’s importance. What we’ve also seen is that smashes don’t tell the entire story–we learn more about a player’s overall ability when we widen our view to smash opportunities.

Smashes in the women’s game

Contemporary women hit far fewer smashes than men do, and they win points less often when they hit them. Despite the differences, the reasoning outlined above applies just as well to the WTA. Let’s take a look.

In the WTA of this decade, smashes result in winners (or induced forced errors) 63% of the time, and smashes result in points won about 75% of the time. Both numbers are lower than the equivalent ATP figures (69% and 81%, respectively), but not dramatically so. Here are the rates of winners, errors, and points won per smash for the 14 women with at least 80 smashes in the MCP dataset:

PLAYER               W/SM  E/SM  PTS/SM  
Jelena Jankovic       73%    9%     83%  
Serena Williams       72%   13%     81%  
Steffi Graf           61%    9%     81%  
Svetlana Kuznetsova   70%   10%     79%  
Simona Halep          66%   11%     76%  
Caroline Wozniacki    61%   16%     74%  
Karolina Pliskova     62%   18%     74%  
Agnieszka Radwanska   54%   13%     74%  
Angelique Kerber      57%   15%     72%  
Martina Navratilova   54%   13%     71%  
Monica Niculescu      50%   15%     70%  
Garbine Muguruza      63%   19%     70%  
Petra Kvitova         59%   22%     68%  
Roberta Vinci         58%   14%     68%

Historical shot-by-shot data is less representative for women than for men, so it’s probably safest to assume that trends in smash success rates are similar for men than for women. If that’s true, Steffi Graf’s era is similar to the present, while Martina Navratilova’s prime saw far fewer smashes going for winners or points won.

Where the women’s game really differs from the men’s is the difference between smash opportunities (lobs) and smashes. As we saw above, 72% of ATP smash opportunities result in smashes. In the current WTA, the corresponding figure is less than half that: 35%. Some of the single-player numbers are almost too extreme to be believed: In 12 matches, Catherine Bellis faced 41 lobs and hit 3 smashes; in 29 charted matches, Jelena Ostapenko saw 103 smash opportunities and tried only 10 smashes. A generation ago, the gender difference was tiny: Graf, Martina Hingis, Arantxa Sanchez Vicario, and Monica Seles all hit smashes in at least three-quarters of their opportunities. But among active players, only Barbora Strycova comes in above 70%.

Here are the smash opportunity numbers for the 17 women with at least 150 smash opportunities in the MCP dataset. SM/SMO is smashes per chance, W/SMO is winners (and induced forced errors) per smash opportunity, E/SMO is errors per opportunity, and PTS/SMO is points won per smash opportunity:

PLAYER                SM/SMO  W/SMO  E/SMO  PTW/SMO  
Maria Sharapova          12%    57%    11%      76%  
Serena Williams          55%    58%    18%      72%  
Steffi Graf              82%    52%    17%      71%  
Karolina Pliskova        47%    52%    16%      70%  
Simona Halep             14%    41%    11%      69%  
Carla Suarez Navarro     25%    33%     9%      69%  
Eugenie Bouchard         29%    50%    18%      68%  
Victoria Azarenka        35%    52%    17%      67%  
Angelique Kerber         39%    42%    14%      66%  
Garbine Muguruza         43%    51%    18%      66%  
Monica Niculescu         57%    41%    19%      65%  
Petra Kvitova            48%    50%    19%      65%  
Agnieszka Radwanska      44%    42%    18%      65%  
Johanna Konta            30%    47%    21%      64%  
Caroline Wozniacki       36%    44%    18%      64%  
Elina Svitolina          14%    38%    14%      63%  
Martina Navratilova      67%    42%    26%      58%

It’s clear from the top of this list that women’s tennis is a different ballgame. Maria Sharapova almost never opts for an overhead, but when faced with a lob, she is the best of them all. Next up is Serena Williams, who hits almost as many smashes as any active player on this list, and is nearly as successful. Recall that in the men’s game, there is a modest positive correlation between smashes per opportunity and points won per smash opportunity; here, the relationship is weaker, and slightly negative.

Because most women hit so few smashes, there isn’t quite as much to be gained by using point probability added (PPA) to measure WTA smash skill. Graf was exceptionally good, comparable to Tsonga in the value she extracted from her smash, but among active players, only Serena and Victoria Azarenka can claim a smash that is worth close to one point per thousand. At the other extreme, Monica Niculescu is nearly as bad as Graf was good, suggesting she ought to figure out a way to respond to more smash opportunities with her signature forehand slice.

Here is the same group of women (minus Navratilova, whose era makes PPA comparisons misleading), with their PPA per 100 points for smashes (SM PPA/100) and smash opportunities (SMO PPA/100):

PLAYER                SM PPA/100  SMO PPA/100  
Maria Sharapova             0.03         0.21  
Serena Williams             0.09         0.15  
Steffi Graf                 0.15         0.14  
Karolina Pliskova          -0.01         0.09  
Carla Suarez Navarro        0.04         0.08  
Simona Halep                0.00         0.07  
Eugenie Bouchard           -0.02         0.03  
Victoria Azarenka           0.08         0.00  
Angelique Kerber           -0.03        -0.02  
Garbine Muguruza           -0.07        -0.03  
Petra Kvitova              -0.07        -0.04  
Monica Niculescu           -0.13        -0.06  
Caroline Wozniacki         -0.01        -0.07  
Agnieszka Radwanska        -0.02        -0.07  
Johanna Konta              -0.12        -0.08  
Elina Svitolina             0.01        -0.09

The table is sorted by smash opportunity PPA, which tells us about a much more relevant skill in the women’s game. Sharapova’s lob-responding ability is well ahead of the pack, worth better than one point above average per 500, with Serena and Graf not far behind. The overall range among these well-studied players, from Sharapova’s 0.21 to Elina Svitolina’s -0.09, is somewhat smaller than the equivalent range in the ATP, but with such outliers as Sharapova here and Delpo on the men’s side, it’s tough to draw firm conclusions from small subsets of players, however elite they are.

Final thought

The approach I’ve outlined here to measure the impact of smash and smash-opportunity skills is one that could be applied to other shots. Smashes are a good place to start because they are so simple: Many of them end points, and even when they don’t, they often virtually guarantee that one player will win the point. While smashes are a bit more complex than they first appear, the complications involved in applying a similar algorithm to, say, backhands and backhand opportunities, are considerably greater. That said, I believe this algorithm represents a promising entry point to these more daunting problems.

Measuring the Impact of the Serve in Men’s Tennis

By just about any measure, the serve is the most important shot in tennis. In men’s professional tennis, with its powerful deliveries and short points, the serve is all the more crucial. It is the one shot guaranteed to occur in every rally, and in many points, it is the only shot.

Yet we don’t have a good way of measuring exactly how important it is. It’s easy to determine which players have the best serves–they tend to show up at the top of the leaderboards for aces and service points won–but the available statistics are very limited if we want a more precise picture. The ace stat counts only a subset of those points decided by the serve, and the tally of service points won (or 1st serve points won, or 2nd serve points won) combines the effect of the serve with all of the other shots in a player’s arsenal.

Aces are not the only points in which the serve is decisive, and some service points won are decided long after the serve ceases to have any relevance to the point. What we need is a method to estimate how much impact the serve has on points of various lengths.

It seems like a fair assumption that if a server hits a winner on his second shot, the serve itself deserves some of the credit, even if the returner got it back in play. In any particular instance, the serve might be really important–imagine Roger Federer swatting away a weak return from the service line–or downright counterproductive–think of Rafael Nadal lunging to defend against a good return and hitting a miraculous down-the-line winner. With the wide variety of paths a tennis point can follow, though, all we can do is generalize. And in the aggregate, the serve probably has a lot to do with a 3-shot rally. At the other extreme, a 25-shot rally may start with a great serve or a mediocre one, but by the time by the point is decided, the effect of the serve has been canceled out.

With data from the Match Charting Project, we can quantify the effect. Using about 1,200 tour-level men’s matches from 2000 to the present, I looked at each of the server’s shots grouped by the stage of the rally–that is, his second shot, his third shot, and so on–and calculated how frequently it ended the point. A player’s underlying skills shouldn’t change during a point–his forehand is as good at the end as it is at the beginning, unless fatigue strikes–so if the serve had no effect on the success of subsequent shots, players would end the point equally often with every shot.

Of course, the serve does have an effect, so points won by the server end much more frequently on the few shots just after the serve than they do later on. This graph illustrates how the “point ending rate” changes:

On first serve points (the blue line), if the server has a “makeable” second shot (the third shot of the rally, “3” on the horizontal axis, where “makeable” is defined as a shot that results in an unforced error or is put back in play), there is a 28.1% chance it ends the point in the server’s favor, either with a winner or by inducing an error on the next shot. On the following shot, the rate falls to 25.6%, then 21.8%, and then down into what we’ll call the “base rate” range between 18% and 20%.

The base rate tells us how often players are able to end points in their favor after the serve ceases to provide an advantage. Since the point ending rate stabilizes beginning with the fifth shot (after first serves), we can pinpoint that stage of the rally as the moment–for the average player, anyway–when the serve is no longer an advantage.

As the graph shows, second serve points (shown with a red line) are a very different story. It appears that the serve has no impact once the returner gets the ball back in play. Even that slight blip with the server’s third shot (“5” on the horizontal axis, for the rally’s fifth shot) is no higher than the point ending rate on the 15th shot of first-serve rallies. This tallies with the conclusions of some other research I did six years ago, and it has the added benefit of agreeing with common sense, since ATP servers win only about half of their second serve points.

Of course, some players get plenty of positive after-effects from their second serves: When John Isner hits a second shot on a second-serve point, he finishes the point in his favor 30% of the time, a number that falls to 22% by his fourth shot. His second serve has effects that mirror those of an average player’s first serve.

Removing unforced errors

I wanted to build this metric without resorting to the vagaries of differentiating forced and unforced errors, but it wasn’t to be. The “point-ending” rates shown above include points that ended when the server’s opponent made an unforced error. We can argue about whether, or how much, such errors should be credited to the server, but for our purposes today, the important thing is that unforced errors aren’t affected that much by the stage of the rally.

If we want to isolate the effect of the serve, then, we should remove unforced errors. When we do so, we discover an even sharper effect. The rate at which the server hits winners (or induces forced errors) depends heavily on the stage of the rally. Here’s the same graph as above, only with opponent unforced errors removed:

The two graphs look very similar. Again, the first serve loses its effect around the 9th shot in the rally, and the second serve confers no advantage on later shots in the point. The important difference to notice is the ratio between the peak winner rate and the base rate, which is now just above 10%. When we counted unforced errors, the ratio between peak and base rate was about 3:2. With unforced errors removed, the ratio is close to 2:1, suggesting that when the server hits a winner on his second shot, the serve and the winner contributed roughly equally to the outcome of the point. It seems more appropriate to skip opponent unforced errors when measuring the effect of the serve, and the resulting 2:1 ratio jibes better with my intuition.

Making a metric

Now for the fun part. To narrow our focus, let’s zero in on one particular question: What percentage of service points won can be attributed to the serve? To answer that question, I want to consider only the server’s own efforts. For unreturned serves and unforced errors, we might be tempted to give negative credit to the other player. But for today’s purposes, I want to divvy up the credit among the server’s assets–his serve and his other shots–like separating the contributions of a baseball team’s pitching from its defense.

For unreturned serves, that’s easy. 100% of the credit belongs to the serve.

For second serve points in which the return was put in play, 0% of the credit goes to the serve. As we’ve seen, for the average player, once the return comes back, the server no longer has an advantage.

For first-serve points in which the return was put in play and the server won by his fourth shot, the serve gets some credit, but not all, and the amount of credit depends on how quickly the point ended. The following table shows the exact rates at which players hit winners on each shot, in the “Winner %” column:

Server's…  Winner %  W%/Base  Shot credit  Serve credit  
2nd shot      21.2%     1.96        51.0%         49.0%  
3rd shot      18.1%     1.68        59.6%         40.4%  
4th shot      13.3%     1.23        81.0%         19.0%  
5th+          10.8%     1.00       100.0%          0.0%

Compared to a base rate of 10.8% winners per shot opportunity, we can calculate the approximate value of the serve in points that end on the server’s 2nd, 3rd, and 4th shots. The resulting numbers come out close to round figures, so because these are hardly laws of nature (and the sample of charted matches has its biases), we’ll go with round numbers. We’ll give the serve 50% of the credit when the server needed only two shots, 40% when he needed three shots, and 20% when he needed four shots. After that, the advantage conferred by the serve is usually canceled out, so in longer rallies, the serve gets 0% of the credit.

Tour averages

Finally, we can begin the answer the question, What percentage of service points won can be attributed to the serve? This, I believe, is a good proxy for the slipperier query I started with, How important is the serve?

To do that, we take the same subset of 1,200 or so charted matches, tally the number of unreturned serves and first-serve points that ended with various numbers of shots, and assign credit to the serve based on the multipliers above. Adding up all the credit due to the serve gives us a raw number of “points” that the player won thanks to his serve. When we divide that number by the actual number of service points won, we find out how much of his service success was due to the serve itself. Let’s call the resulting number Serve Impact, or SvI.

Here are the aggregates for the entire tour, as well as for each major surface:

         1st SvI  2nd SvI  Total SvI  
Overall    63.4%    31.0%      53.6%  
Hard       64.6%    31.5%      54.4%  
Clay       56.9%    27.0%      47.8%  
Grass      70.8%    37.3%      61.5%

Bottom line, it appears that just over half of service points won are attributable to the serve itself. As expected, that number is lower on clay and higher on grass.

Since about two-thirds of the points that men win come on their own serves, we can go even one step further: roughly one-third of the points won by a men’s tennis player are due to his serve.

Player by player

These are averages, and the most interesting players rarely hew to the mean. Using the 50/40/20 multipliers, Isner’s SvI is a whopping 70.8% and Diego Schwartzman‘s is a mere 37.7%. As far from the middle as those are, they understate the uniqueness of these players. I hinted above that the same multipliers are not appropriate for everyone; the average player reaps no positive after-effects of his second serve, but Isner certainly does. The standard formula we’ve used so far credits Isner with an outrageous SvI, even without giving him credit for the “second serve plus one” points he racks up.

In other words, to get player-specific results, we need player-specific multipliers. To do that, we start by finding a player-specific base rate, for which we’ll use the winner (and induced forced error) rate for all shots starting with the server’s fifth shot on first-serve points and shots starting with the server’s fourth on second-serve points. Then we check the winner rate on the server’s 2nd, 3rd, and 4th shots on first-serve points and his 2nd and 3rd shots on second-serve points, and if the rate is at least 20% higher than the base rate, we give the player’s serve the corresponding amount of credit.

Here are the resulting multipliers for a quartet of players you might find interesting, with plenty of surprises already:

                   1st serve              2nd serve       
                    2nd shot  3rd  4th     2nd shot  3rd  
Roger Federer            55%  50%  30%           0%   0%  
Rafael Nadal             31%   0%   0%           0%   0%  
John Isner               46%  41%   0%          34%   0%  
Diego Schwartzman        20%  35%   0%           0%  25%  
Average                  50%  30%  20%           0%   0%

Roger Federer gets more positive after-effects from his first serve than average, more even than Isner does. The big American is a tricky case, both because so few of his serves come back and because he is so aggressive at all times, meaning that his base winner rate is very high. At the other extreme, Schwartzman and Rafael Nadal get very little follow-on benefit from their serves. Schwartzman’s multipliers are particularly intriguing, since on both first and second serves, his winner rate on his third shot is higher than on his second shot. Serve plus two, anyone?

Using player-specific multipliers makes Isner’s and Schwartzman’s SvI numbers more extreme. Isner’s ticks up a bit to 72.4% (just behind Ivo Karlovic), while Schwartzman’s drops to 35.0%, the lowest of anyone I’ve looked at. I’ve calculated multipliers and SvI for all 33 players with at least 1,000 tour-level service points in the Match Charting Project database:

Player                 1st SvI  2nd SvI  Total SvI  
Ivo Karlovic             79.2%    56.1%      73.3%  
John Isner               78.3%    54.3%      72.4%  
Andy Roddick             77.8%    51.0%      71.1%  
Feliciano Lopez          83.3%    37.1%      68.9%  
Kevin Anderson           77.7%    42.5%      68.4%  
Milos Raonic             77.4%    36.0%      66.0%  
Marin Cilic              77.1%    34.1%      63.3%  
Nick Kyrgios             70.6%    41.0%      62.5%  
Alexandr Dolgopolov      74.0%    37.8%      61.3%  
Gael Monfils             69.8%    37.7%      60.8%  
Roger Federer            70.6%    32.0%      58.8%  
Player                 1st SvI  2nd SvI  Total SvI  
Bernard Tomic            67.6%    28.7%      58.5%  
Tomas Berdych            71.6%    27.0%      57.2%  
Alexander Zverev         65.4%    30.2%      54.9%  
Fernando Verdasco        61.6%    32.9%      54.3%  
Stan Wawrinka            65.4%    33.7%      54.2%  
Lleyton Hewitt           66.7%    32.1%      53.4%  
Juan Martin Del Potro    63.1%    28.2%      53.4%  
Grigor Dimitrov          62.9%    28.6%      53.3%  
Jo Wilfried Tsonga       65.3%    25.9%      52.7%  
Marat Safin              68.4%    22.7%      52.3%  
Andy Murray              63.4%    27.5%      52.0%  
Player                 1st SvI  2nd SvI  Total SvI  
Dominic Thiem            60.6%    28.9%      50.8%  
Roberto Bautista Agut    55.9%    32.5%      49.5%  
Pablo Cuevas             57.9%    28.9%      47.8%  
Richard Gasquet          56.0%    29.0%      47.5%  
Novak Djokovic           56.0%    26.8%      47.3%  
Andre Agassi             54.3%    31.4%      47.1%  
Gilles Simon             55.7%    28.4%      46.7%  
Kei Nishikori            52.2%    30.8%      45.2%  
David Ferrer             46.9%    28.2%      41.0%  
Rafael Nadal             42.8%    27.1%      38.8%  
Diego Schwartzman        39.5%    25.8%      35.0%

At the risk of belaboring the point, this table shows just how massive the difference is between the biggest servers and their opposites. Karlovic’s serve accounts for nearly three-quarters of his success on service points, while Schwartzman’s can be credited with barely one-third. Even those numbers don’t tell the whole story: Because Ivo’s game relies so much more on service games than Diego’s does, it means that 54% of Karlovic’s total points won–serve and return–are due to his serve, while only 20% of Schwartzman’s are.

We didn’t need a lengthy analysis to show us that the serve is important in men’s tennis, or that it represents a much bigger chunk of some players’ success than others. But now, instead of asserting a vague truism–the serve is a big deal–we can begin to understand just how much it influences results, and how much weak-serving players need to compensate just to stay even with their more powerful peers.

The Negative Impact of Time of Court

With 96 men’s matches in the books so far at Roland Garros this year, we’ve seen only one go to the absolute limit, past 6-6 in the fifth set. Still, we’ve had our share of lengthy, brutal five-set fights, including three matches in the first round that exceeded the four-hour mark. The three winners of those battles–Victor Estrella, David Ferrer, and Rogerio Dutra Silva–all fell to their second-round opponent.

A few years ago, I identified a “hangover effect” after Grand Slam marathons, defined as those matches that reach 6-6 in the fifth. Players who emerge victorious from such lengthy struggles would often already be considered underdogs in their next matches–after all, elite players rarely need to work so hard to advance–but marathon winners underperform even when we take their underdog status into account. (Earlier this week, I showed that women suffer little or no hangover effect after marathon third sets.)

A number of readers suggested I take a broader look at the effect of match length. After all, there are plenty of slugfests that fall just short of the marathon threshold, and some of those, like Ferrer’s loss yesterday to Feliciano Lopez, 6-4 in the final set, are more physically testing than some of those that reach 6-6. Match time still isn’t a perfect metric for potential fatigue–a four-hour match against Ferrer is qualitatively different from four hours on court with Ivo Karlovic–but it’s the best proxy we have for a very large sample of matches.

What happens next?

I took over 7,200 completed men’s singles matches from Grand Slams back to 2001 and separated them into groups by match time: one hour to 1:29, 1:30 to 2:00, and so on, up to a final category of 4:30 and above. Then I looked at how the winners of all those matches fared against their next opponents:

Prev Length   Matches  Wins  Win %  
1:00 to 1:29      448   275  61.4%  
1:30 to 1:59     1918  1107  57.7%  
2:00 to 2:29     1734   875  50.5%  
2:30 to 2:59     1384   632  45.7%  
3:00 to 3:29      976   430  44.1%  
3:30 to 3:59      539   232  43.0%  
4:00 to 4:29      188    64  34.0%  
4:30 and up        72    23  31.9%

The trend couldn’t be any clearer. If the only thing you know about a Slam matchup is how long the players spent on court in their previous match, you’d bet on the guy who recorded his last win in the shortest amount of time.

Of course, we know a lot more about the players than that. Andy Murray spent 3:34 on court yesterday, but even with his clay-court struggles this year, we would favor him in the third round against most of the men in the draw. As I’ve done in previous studies, let’s account for overall player skill by estimating the probability of each player winning each of these 7,200+ matches. Here are the same match-length categories, with “expected wins” (based on surface-specific Elo, or sElo) shown as well:

Prev Length   Wins  Exp Wins  Exp Win %  Ratio  
1:00 to 1:29   275       258      57.5%   1.07  
1:30 to 1:59  1107      1058      55.2%   1.05  
2:00 to 2:29   875       881      50.8%   0.99  
2:30 to 2:59   632       657      47.5%   0.96  
3:00 to 3:29   430       445      45.6%   0.97  
3:30 to 3:59   232       244      45.3%   0.95  
4:00 to 4:29    64        77      41.2%   0.83  
4:30 and up     23        30      42.1%   0.76

Again, there’s not much ambiguity in the trend here. Better players spend less time on court, so if you know someone beat their previous opponent in 1:14, you can infer that he’s a very good player. Often that assumption is wrong, but in the aggregate, it holds up.

The “Ratio” column shows the relationship between actual winning percentage (from the first table) and expected winning percentage. If previous match time had no effect, we’d expect to see ratios randomly hovering around 1. Instead, we see a steady decline from 1.07 at the top–meaning that players coming off of short matches win 7% more often than their skill level would otherwise lead us to forecast–to 0.76 at the bottom, indicating that competitors tend to underperform following a battle of 4:30 or longer.

It’s difficult to know whether we’re seeing a direct effect of time of court or a proxy for form. As good as surface-specific Elo ratings are, they don’t capture everything that could possibly predict the outcome of a match, especially micro-level considerations like a player’s comfort on a specific type of surface or at a certain tournament. sElo also needs a little time to catch up with players making fast improvements, particularly when they are very young. All this is to say that our correction for overall skill level will never be perfect.

Thus, a 75-minute win may improve a player’s chances by keeping him fresh for the next round … or it might tell us that–for whatever reason–he’s a stronger competitor right now than our model gives him credit for. One point in favor of the latter is that, at the most extreme, less time on court doesn’t help: Players don’t appear to benefit from advancing via walkover. That isn’t a slam-dunk argument–some commentators believe that walkovers could be detrimental due to the long resulting layoff at a Slam–but it does show us that less time on court isn’t always a positive.

Whatever the underlying cause, we can tweak our projections accordingly. Murray could be a little weaker than usual tomorrow after his length battle yesterday with Martin Klizan. Albert Ramos, the only man to complete a second-rounder in less than 90 minutes, might be playing a bit better than his rating suggest. It’s certainly evident that match time has something to tell us even when players aren’t stretched to the breaking point of a marathon fifth set.

Angelique Kerber’s Unclutch Unforced Errors

It’s been a rough year for Angelique Kerber. Despite her No. 1 WTA ranking and place at the top of the French Open draw, she lost her opening match on Sunday against the unseeded Ekaterina Makarova. Adding insult to injury, the loss goes down in the record books as a lopsided-looking 6-2 6-2.

Andrea Petkovic chimed in with her diagnosis of Kerber’s woes:

She’s simply playing without confidence right now. It was tight, even though the scoreline was 2 and 2 but everyone who knows a thing about tennis knew that Angie made errors whenever it mattered because she’s playing without any confidence right now – errors she didn’t make last year.

This is one version of a common analysis: A player lost because she crumbled on the big points. While that probably doesn’t cover all of Kerber’s issues on Sunday–Makarova won 72 points to her 55–it is true that big points have a disproportionate effect on the end result. For every player who squanders a dozen break points yet still wins the match, there are others who falter at crucial moments and ultimately lose.

This family of theories–that a player over- or under-performed at big moments–is testable. For instance, I showed last summer that Roger Federer’s Wimbledon loss to Milos Raonic was due in part to his weaker performance on more important points. We can do the same with Kerber’s early exit.

Here’s how it works. Once we calculate each player’s probability of winning the match before each point, we can assign each point a measure of importance–I prefer to call it leverage, or LEV–that quantifies how much the single point could effect the outcome of the match. At 3-0, 40-0, it’s almost zero. At 3-3, 40-AD in the deciding set, it might be over 10%. Across an entire tournament’s worth of matches, the average LEV is around 5% to 6%.

If Petko is right, we’ll find that the average LEV of Kerber’s unforced errors was higher than on other points. (I’ve excluded points that ended with the serve, since neither player had a chance to commit an unforced error.) Sure enough, Kerber’s 13 groundstroke UEs (that is, excluding double faults) had an average LEV of 5.5%, compared to 3.8% on points that ended some other way. Her UE points were 45% more important than non-UE points.

Let’s put that number in perspective. Among the 86 women for whom I have point-by-point UE data for their first-round matches this week*, ten timed their errors even worse than Kerber did. Magdalena Rybarikova was the most extreme: Her eight UEs against Coco Vandeweghe were more than twice as important, on average, as the rest of the points in that match. Seven of the ten women with bad timing lost their matches, and two others–Agnieszka Radwanska and Marketa Vondrousova–committed so few errors (3 and 4, respectively), that it didn’t really matter. Only Dominika Cibulkova, whose 15 errors were about as badly timed as Kerber’s, suffered from unclutch UEs yet managed to advance.

* This data comes from the Roland Garros website. I aggregate it after each major and make it available here.

Another important reference point: Unforced errors are evenly distributed across all leverage levels. Our instincts might tell us otherwise–we might disproportionately recall UEs that came under pressure—-but the numbers don’t bear it out. Thus, Kerber’s badly timed errors are just as badly timed when we compare her to tour average.

They are also poorly timed when compared to her other recent performances at majors. Petkovic implied as much when she said her compatriot was making “errors she didn’t make last year.” Across her 19 matches at the previous four Slams, her UEs occurred on points that were 11% less important than non-UE points. Her errors caused her to lose relatively more important points in only 5 of the 19 matches, and even in those matches, the ratio of UE leverage to non-UE leverage never exceeded 31%, her ratio in Melbourne this year against Tsurenko. That’s still better than her performance on Sunday.

Across so many matches, a difference of 11% is substantial. Of the 30 players with point-by-point UE data for at least eight matches at the previous four majors, only three did a better job timing their unforced errors. Radwanska heads the list, at 16%, followed by Timea Bacsinszky at 14% and Kiki Bertens at 12%. The other 26 players committed their unforced errors at more important moments than Kerber did.

As is so often the case in tennis, it’s difficult to establish if a stat like this is indicative of a longer-trend trend, or if it is mostly noise. We don’t have point-by-point data for most of Kerber’s matches, so we can’t take the obvious next step of checking the rest of her 2017 matches for similarly unclutch performances. Instead, we’ll have to keep tabs on how well she limits UEs at big moments on those occasions where we have the data necessary to do so.

Bouncing Back From a Marathon Third Set

In this year’s edition of the French Open, we’ve already seen two women’s matches charge past the 6-6 mark in the third set. On Sunday, Madison Brengle outlasted Julia Goerges 13-11 in the decider, and yesterday, Kristina Mladenovic overcame Jennifer Brady 9-7 in the final set. Marathon three-setters aren’t as gut-busting as the five-set equivalent on the men’s tour, yet they still require players to go beyond the usual limit of a tour match.

Do marathon three-setters affect the fortunes of those players that move on to the next round? Back in 2012, I published a study showing that men who win marathon five-setters (that is, matches that go to 8-6 or longer) win fewer than 30% of their following matches, a rate far worse than what we would expect, given the quality of their next opponents. It seems likely that long three-setters wouldn’t have the same effect, especially since many top women are willing to play five-setters themselves.

The numbers bear out the intuition. From 2001 to the 2017 Australian Open, there have been 185 marathon three-setters in Grand Slam main draws, and the winners of those matches have gone on to win 42.2% of their next contests. That’s more than the equivalent number for men, and it’s even better than it sounds.

Players who need to go deep into a third set to vanquish an early-round opponent are, on average, weaker than those who win in straight sets, so many of the marathon women would already be considered underdogs in their next matches. Using sElo–surface-specific Elo, which I recently introduced–we see that these 185 marathon women would have been expected to win only 44.0% of their following matches. There may be a real effect here, but it is a minor one, especially compared to the fortunes of players who struggle through marathon five-setters.

I ran the same algorithm for women’s Slam matches that ended at 7-6, 7-5, and 6-4 or 6-3 in the final set. Since only the US Open uses the third-set tiebreak format, the available sample for that score is limited, which may explain a slightly wacky result. For the other scores, we see numbers that are roughly similar to the marathon findings. Winners tend to be underdogs against their next opponents, but there is little, if any, hangover effect:

3rd Set Score  Sample  Next W%  Next ExpW%  
Marathons         185    42.2%       44.0%  
7-6                56    48.2%       42.2%  
7-5               232    43.1%       42.7%  
6-4 / 6-3         421    41.6%       43.2%

In short: A long match often tells us something about the winner’s chances against her next foe, but it’s something that we already knew. The tight three-setter itself–marathon or otherwise–has little effect on her chances later on. That’s good news for Mladenovic, who will be back on court tomorrow against Sara Errani, an opponent likely to give her another grueling workout.

Diego Schwartzman’s Return Game Is Even Better Than I Thought

Diego Schwartzman is one of the most unusual players on the ATP tour. Even shorter than David Ferrer, his serve will never be a weapon, so the only way he can compete is by neutralizing everyone else’s offerings and winning baseline battles. Up to No. 34 in this week’s official rankings and No. 35 on the Elo list, he’s proven he can do that against some very good players.

Using the ATP stats leaderboard at Tennis Abstract, we can get a quick sense of how his return game compares with the elites. At tour level in the last 52 weeks (through Monte Carlo), he ranks third with 42.3% return points won, behind only Andy Murray and Novak Djokovic. He is particularly effective against second serves, winning 56.6% of those, better than anyone else on tour. He has broken in 31.8% of his return games, another third-place showing, this time behind Murray and Rafael Nadal.

Yet the leaderboard warns us to tread carefully. In the last year, Murray’s opponents have been far superior to Schwartzman’s, with a median rank of 24 and a mean rank of 41.5. The Argentine’s opponents have rated at 45.5 and 54.8, respectively. Murray, Djokovic, and Nadal are far better all-around players than Schwartzman, so they regularly reach later rounds, where the quality of competition goes way up.

Competition quality is one of the knottiest aspects of tennis analytics, and it is far from being solved. If we want to compare Murray to Djokovic, competition quality isn’t such a big factor. One or the other might get lucky over a span of months, but in the long run, the two best players on tour will face roughly equivalent levels of competition. But when we expand our view to players like Schwartzman–or even a top-tenner such as Dominic Thiem–we can no longer assume that opponent quality will even out. To use a term from other sports, the ATP has a very unbalanced schedule, and the schedule is always more challenging for the best players.

Correcting for competition quality is also key to understanding how any particular player evolves over time. If a player’s results improve, he’ll usually start facing more challenging competition, as Schwartzman is doing this spring in his first shot at the full slate of clay-court Masters events. If his return numbers decline, is he actually playing worse, or is he simply competing at his past level against tougher opponents?

Adjusting for competition

To properly compare players, we need to identify similarities in their schedules. Any pair of tour regulars have played many of the same opponents, even if they’ve never played each other. For instance, since the beginning of last season, Murray and Djokovic have faced 18 of the same players–some more than once. Further down the ranking list, players tend to have fewer opponents in common, but as we’ll see, that’s an obstacle we can overcome.

Here’s how the adjustment works: For a pair of players, find all the opponents both men have faced on the same surface. For example, both Murray and Djokovic have played David Goffin on clay in the last 16 months. Murray won 53.7% of clay return points against the Belgian, while Djokovic won only 42.1%, meaning that Djokovic returned about 22% worse than Murray did. We repeat the process for every surface-player combination, weight the results so that longer matches (or larger numbers of matches) count more heavily, and find the average.

When we do that for the top two men, we find that Djokovic has returned 2.3% better. (That’s a percentage, not percentage points. A great returner wins about 40% of return points, and a 2.3% improvement on that is roughly 41%.) Our finding suggests that Murray has faced somewhat weaker-serving competition: Since the beginning of 2016, he has won 42.9% of return points, compared to Djokovic’s 43.3%–a smaller gap than the competition-adjusted one.

It takes more work to reliably compare someone like Schwartzman to the elites, since their schedules overlap so much less. So before adjusting Diego’s return numbers, we’ll take several intermediate steps. Let’s start with the world No. 3 Stanislas Wawrinka. We follow the above process twice: Once for Wawrinka and Murray, then again for Stan and Novak. Run the numbers, and we find that Wawrinka’s return game is 22.5% weaker than Murray’s and 24.3% weaker than Djokovic’s. Wawrinka’s rates relative to the other two players correspond very well with what we already found, suggesting that Djokovic is a little better than his rival. Weighting the two numbers by sample size–which, in this case, is almost identical–we slightly adjust those two comparisons and conclude that Wawrinka’s return game is 22.4% worse than Murray’s.

Generating competition-adjusted numbers for each subsequent player follows the same pattern. For No. 4 Federer, we run the algorithm three times, one for each of the players ranked above him, then we aggregate the results. For No. 34 Schwartzman, we go through the process 33 times. Thanks to the magic of computers, it takes only a few seconds to adjust 16 months worth of return stats for the ATP top 50.

Below are the results for 2016-17. Players are ranked by “relative return points won” (REL RPW), where a rating of 1.0 is arbitrarily given to Murray, and a rating of 0.98 means that a player wins 2% fewer return points than Murray against equivalent opposition. The “EX RPW” column puts those numbers in a more familiar context: The top-ranked player’s rating is set equal to 43.0%–approximately the best RPW of any player in the last few seasons–and everyone else’s is adjusted accordingly.  The last two columns show each player’s actual rate of return points won and their rank among the ATP top 50:

1     Diego Schwartzman         1.04   43.0%   42.4%     4  
2     Novak Djokovic            1.02   42.1%   43.3%     1  
3     Andy Murray               1.00   41.2%   42.9%     2  
4     Rafael Nadal              0.98   40.3%   42.6%     3  
5     David Goffin              0.97   40.1%   41.3%     5  
6     Gilles Simon              0.96   39.6%   40.1%     9  
7     Kei Nishikori             0.95   39.3%   40.1%    10  
8     David Ferrer              0.95   39.1%   40.6%     7  
9     Roger Federer             0.94   38.7%   38.7%    15  
10    Gael Monfils              0.93   38.5%   39.8%    11  

11    Roberto Bautista Agut     0.93   38.3%   40.3%     8  
12    Ryan Harrison             0.92   37.9%   36.7%    33  
13    Richard Gasquet           0.92   37.9%   40.8%     6  
14    Daniel Evans              0.91   37.6%   36.9%    27  
15    Juan Martin Del Potro     0.91   37.5%   36.8%    32  
16    Benoit Paire              0.90   37.0%   38.1%    19  
17    Mischa Zverev             0.90   36.9%   36.9%    28  
18    Grigor Dimitrov           0.89   36.4%   38.2%    18  
19    Fabio Fognini             0.88   36.4%   39.7%    12  
20    Fernando Verdasco         0.88   36.4%   38.3%    16  

21    Joao Sousa                0.88   36.2%   38.3%    17  
22    Dominic Thiem             0.88   36.2%   38.1%    20  
23    Stani Wawrinka            0.88   36.1%   37.5%    22  
24    Alexander Zverev          0.88   36.0%   37.5%    23  
25    Albert Ramos              0.87   35.9%   38.9%    14  
26    Kyle Edmund               0.86   35.5%   36.1%    37  
27    Jack Sock                 0.86   35.5%   36.6%    34  
28    Viktor Troicki            0.86   35.4%   37.1%    26  
29    Marin Cilic               0.86   35.4%   37.3%    25  
30    Pablo Carreno Busta       0.86   35.3%   39.4%    13  

31    Milos Raonic              0.86   35.2%   36.1%    38  
32    Pablo Cuevas              0.85   35.1%   36.9%    29  
33    Tomas Berdych             0.85   35.1%   36.9%    30  
34    Borna Coric               0.85   34.9%   36.1%    39  
35    Nick Kyrgios              0.85   34.9%   35.7%    41  
36    Philipp Kohlschreiber     0.84   34.7%   37.9%    21  
37    Jo Wilfried Tsonga        0.84   34.6%   36.2%    36  
38    Sam Querrey               0.83   34.3%   34.6%    44  
39    Lucas Pouille             0.82   33.9%   36.9%    31  
40    Feliciano Lopez           0.81   33.2%   35.2%    43  

41    Robin Haase               0.80   33.0%   36.1%    40  
42    Paolo Lorenzi             0.80   32.9%   37.5%    24  
43    Donald Young              0.78   32.2%   36.3%    35  
44    Bernard Tomic             0.78   32.1%   34.1%    45  
45    Nicolas Mahut             0.76   31.4%   35.4%    42  
46    Steve Johnson             0.75   31.0%   33.8%    46  
47    Florian Mayer             0.74   30.3%   33.5%    47  
48    John Isner                0.73   30.0%   29.8%    49  
49    Gilles Muller             0.72   29.8%   32.4%    48  
50    Ivo Karlovic              0.63   25.9%   26.4%    50

The big surprise: Schwartzman is number one! While the average ranking of his opponents was considerably lower than that of the elites, it appears that he has faced bigger-serving opponents than have Murray or Djokovic. The top five on this list–Schwartzman, Murray, Djokovic, Nadal, and Goffin–do not force any major re-evaluation of who we consider to be the game’s best returners, but the competition-adjusted metric does offer more evidence that Schwartzman really belongs there.

There is a similar predictability at the bottom of the list. The five players rated the worst by the competition-adjusted metric–Steve Johnson, Florian Mayer, John Isner, Gilles Muller, and Ivo Karlovic–are the same five who sit at the bottom of the actual RPW ranking, with only Isner and Muller swapping places. This degree of consistency at the top and bottom of the list is reassuring: The metric is correcting for something important, but it isn’t spitting out any truly crazy results.

There are, however, some surprises. Three players do very well when their return games are adjusted for competition: Ryan Harrison, Daniel Evans, and Juan Martin del Potro, all of whom jump from the bottom half to the top 15. In a sense, this is a surface adjustment for Harrison and Evans, both of whom have played almost exclusively on hard courts. Players win fewer return points on faster surfaces (and faster surfaces attract bigger-serving competitors, magnifying the effect), so when adjusted for competition, someone who plays only on hard courts will see his numbers improve. Del Potro, on the other hand, has been absolutely hammered by tough competition, so in his case the correction is giving him credit for the difficult opponents he has had to face.

Several clay court specialists find their return stats adjusted in the wrong direction. Last week’s finalist, Albert Ramos, falls from 14th to 25th, Pablo Carreno Busta drops from 13th to 30th, and Roberto Bautista Agut and Paolo Lorenzi see their numbers take a hit as well. This is the reverse of the effect that pushed Harrison and Evans up the list: Clay-court specialists spend more time on the dirt and they play against weaker-serving opponents, so their season averages make them look like better returners than they really are. It appears that these players are all particularly bad on hard courts: When I ran the algorithm with only clay-court results, Bautista Agut, Ramos, and Carreno Busta all appeared among the top 12 in competition-adjusted return points won. It’s their abysmal hard-court performances that pull down their longer-term numbers.

Beyond RPW

This algorithm–or something like it–has a great deal of potential beyond simply correcting return points won for tour-level competition quality. It could be used for any stat, and if competition-adjusted return rates were combined with corrected rates of service points won, it would generate a plausible overall player rating system.

Such a rating system would be more valuable if the algorithm were extended to players beyond the top 50, as well. Just as Schwartzman doesn’t yet have that many common opponents with the elites, Challenger-level stalwarts don’t have share many opponents with tour regulars. But there is enough overlap that, when combining the shared opponents of dozens of players, we might be able to get a better grip on how Challenger-level competition compares to that of the highest levels. Essentially, we can compare adjacent levels–the elites to the middle of the pack (say, ATP ranks 21 to 50), the middle of the pack to the next 50, and so on–to get a more comprehensive idea of how much players must improve to achieve certain goals.

Finally, adjusting serve and return stats so that we have a set of competition-neutral numbers for every player, for each season of his career, we will gain a clearer picture of which players are improving and by how much. Official rankings and Elo ratings tell us a lot, but they are sometimes fooled by lucky breaks, close wins, or inconsistent opposition. And they cannot isolate individual stats, which may be particularly useful for developmental purposes.

Adjusting for opposition quality is standard practice for analysts of many other sports, and it will help tennis analytics move forward as well. If nothing else, it has shown us that one extreme performance–Schwartzman’s return game–is much more than a fluke, and that service return greatness isn’t limited to the big four.