Digging Out of the Holes of 0-40 and 15-40

In the men’s professional game, serving at 0-40 isn’t a death sentence, but it isn’t a good place to be. An average player wins about 65% of service points, and at that rate, his chance of coming back from 0-40 is just a little better than one in five.

Some players are better than others at executing this sort of comeback. Tommy Robredo, for instance, has come back from 0-40 nearly 60% more often than we’d expect, while Sam Querrey digs out of the 0-40 hole one-third less often than we would predict.

Measuring a player’s success rate in these scenarios isn’t simply a matter of counting up 0-40 games. That’s what we saw on the ATP official site last week, and it’s woefully inadequate. That article marvels at Ivo Karlovic‘s “clutch” accomplishments from 0-40 and 15-40, when we could easily have guessed that Ivo would lead just about any serving category. Big serving isn’t clutch if it’s what you always do.

Statistics are only valuable in context, and that is particularly true in tennis. Simply counting 0-40 games and reporting the results hides a huge amount of potential insight. Whether a player wins or loses (a game, a set, a match, or a stretch of matches) is only the first question. To deliver any kind of meaningful analysis, we need to adjust those results for the competition and consider what we already know about the players we’re studying.

Rather than tear apart that article, though, let’s do the analysis correctly.

The number of times a player comes back from 0-40 or 15-40 isn’t what’s important. As we’ve seen, big servers will dominate those categories. That doesn’t tell us who is particularly effective (or, dare we say, “clutch”) in such a situation, it only identifies the best servers. What matters is how often players come back compared to how often we would expect them to, taking into consideration their serving ability.

Karlovic is an instructive example. Over the last few years–the time span available in this dataset of point-by-point match records–Ivo has gone down 0-40 56 times, holding 17 of those games, a rate of 30.4%. That’s third-best on tour, behind John Isner and Samuel Groth. But compared to how well we would expect Karlovic to serve, he’s only 7% better than neutral, right in the middle of the ATP pack.

Before diving into the results, a few more notes on methodology. For each 0-40 or 15-40 game, I calculated the server’s rate of service points won in that match. Since we would expect 0-40 games to occur more often in matches with good returners, in-match rates seem more accurate than season-long aggregates. Given the in-match rate of serve points won, I then determined the odds that the server would come back from the 0-40 or 15-40 score. For each game, then, we have a result (came back or didn’t come back) and an estimate of the comeback’s likelihood. Combining both numbers for all of a player’s service games tells us how effective he was at these scores.

For 30 of the players best represented in the dataset, here are their results at 0-40, showing the number of games, the number of successful comebacks, the rate of successful comebacks, and the degree to which the player exceeded expectations from 0-40:

Player                  0-40  0-40 W  0-40 W%  W/Exp  
Tommy Robredo            110      30    27.3%   1.59  
Denis Istomin            114      26    22.8%   1.36  
John Isner                87      31    35.6%   1.34  
Guillermo Garcia-Lopez   161      29    18.0%   1.32  
Kevin Anderson           130      38    29.2%   1.28  
Bernard Tomic            110      24    21.8%   1.25  
Fernando Verdasco        141      30    21.3%   1.17  
Rafael Nadal             140      32    22.9%   1.15  
Kei Nishikori            122      23    18.9%   1.15  
Marin Cilic              125      26    20.8%   1.14  
Player                  0-40  0-40 W  0-40 W%  W/Exp  
Jo-Wilfried Tsonga       124      29    23.4%   1.14  
Novak Djokovic           124      34    27.4%   1.12  
Andreas Seppi            145      24    16.6%   1.09  
Grigor Dimitrov          115      22    19.1%   1.08  
Philipp Kohlschreiber    146      28    19.2%   1.08  
Roger Federer            107      26    24.3%   1.07  
Ivo Karlovic              56      17    30.4%   1.07  
Santiago Giraldo         113      18    15.9%   1.06  
Alexandr Dolgopolov      141      25    17.7%   1.03  
Milos Raonic              82      23    28.0%   1.01  
Player                  0-40  0-40 W  0-40 W%  W/Exp  
Tomas Berdych            149      30    20.1%   1.01  
Jeremy Chardy            122      21    17.2%   0.98  
Feliciano Lopez          136      26    19.1%   0.97  
Fabio Fognini            211      24    11.4%   0.97  
Mikhail Youzhny          155      18    11.6%   0.92  
David Ferrer             203      32    15.8%   0.89  
Richard Gasquet          152      25    16.4%   0.87  
Andy Murray              164      24    14.6%   0.80  
Gilles Simon             158      16    10.1%   0.72  
Sam Querrey               84      12    14.3%   0.68

As I mentioned above, Robredo has been incredibly effective in these situations, coming back from 0-40 30 times instead of the 19 times we would have expected. Some big servers, such as Isner and Kevin Anderson, are even better than their well-known weapons would leads us to expect, while others, such as Karlovic and Milos Raonic, aren’t noticeably more effective at 0-40 than they are in general.

Many of these extremes don’t hold up when we turn to the results from 15-40. Quite a few more games reach 15-40 than 0-40, so the more limited variation at 15-40 suggests that many of the extreme results from 0-40 can be ascribed to an inadequate sample. For instance, Robredo–our 0-40 hero–falls to neutral at 15-40. Here is the complete list:

Player                  15-40  15-40 W  15-40 W%  W/Exp  
John Isner                238      122     51.3%   1.33  
Milos Raonic              215       98     45.6%   1.18  
Feliciano Lopez           304      108     35.5%   1.17  
Jo-Wilfried Tsonga        301      119     39.5%   1.17  
Denis Istomin             304      101     33.2%   1.17  
Rafael Nadal              320      118     36.9%   1.16  
Ivo Karlovic              148       68     45.9%   1.15  
Kevin Anderson            338      132     39.1%   1.15  
Guillermo Garcia-Lopez    405      106     26.2%   1.14  
Andreas Seppi             396      113     28.5%   1.12  
Player                  15-40  15-40 W  15-40 W%  W/Exp  
Bernard Tomic             273       86     31.5%   1.12  
Kei Nishikori             298       96     32.2%   1.10  
Novak Djokovic            348      132     37.9%   1.07  
Richard Gasquet           325      106     32.6%   1.07  
Roger Federer             281      109     38.8%   1.07  
Fernando Verdasco         306       94     30.7%   1.06  
Philipp Kohlschreiber     352      110     31.3%   1.06  
Andy Murray               431      135     31.3%   1.06  
Santiago Giraldo          331       86     26.0%   1.05  
Tomas Berdych             398      131     32.9%   1.05  
Player                  15-40  15-40 W  15-40 W%  W/Exp  
Marin Cilic               357      109     30.5%   1.05  
Sam Querrey               244       78     32.0%   1.04  
Jeremy Chardy             300       91     30.3%   1.04  
Fabio Fognini             422       98     23.2%   1.03  
Tommy Robredo             285       78     27.4%   0.99  
Grigor Dimitrov           307       89     29.0%   0.99  
David Ferrer              498      138     27.7%   0.98  
Alexandr Dolgopolov       299       77     25.8%   0.95  
Mikhail Youzhny           339       77     22.7%   0.94  
Gilles Simon              426       93     21.8%   0.91

The big servers are better represented at the top of this ranking. Even though Isner is expected to come back from 15-40 nearly 40% of the time–better than almost anyone on tour–he exceeds that expectation by one-third, far more than anyone else considered here.

Finally, let’s look at comebacks from 0-30:

Player                  0-30  0-30 W  0-30 W%  W/Exp  
John Isner               338     229    67.8%   1.19  
Bernard Tomic            299     146    48.8%   1.15  
Grigor Dimitrov          342     166    48.5%   1.11  
Novak Djokovic           409     235    57.5%   1.10  
Santiago Giraldo         344     142    41.3%   1.10  
Fernando Verdasco        373     175    46.9%   1.10  
Rafael Nadal             376     194    51.6%   1.09  
Tomas Berdych            492     262    53.3%   1.09  
Tommy Robredo            296     132    44.6%   1.08  
Roger Federer            344     193    56.1%   1.08  
Player                  0-30  0-30 W  0-30 W%  W/Exp  
Feliciano Lopez          326     161    49.4%   1.07  
Alexandr Dolgopolov      347     154    44.4%   1.07  
Marin Cilic              378     179    47.4%   1.06  
Jo-Wilfried Tsonga       357     185    51.8%   1.06  
Guillermo Garcia-Lopez   380     146    38.4%   1.06  
Ivo Karlovic             186     118    63.4%   1.04  
Philipp Kohlschreiber    395     185    46.8%   1.03  
Denis Istomin            314     135    43.0%   1.03  
Kei Nishikori            341     145    42.5%   1.03  
David Ferrer             529     227    42.9%   1.02  
Player                  0-30  0-30 W  0-30 W%  W/Exp  
Kevin Anderson           361     181    50.1%   1.02  
Mikhail Youzhny          390     142    36.4%   1.00  
Andy Murray              419     185    44.2%   1.00  
Andreas Seppi            418     164    39.2%   0.99  
Jeremy Chardy            316     132    41.8%   0.99  
Milos Raonic             246     139    56.5%   0.99  
Fabio Fognini            478     153    32.0%   0.99  
Sam Querrey              292     131    44.9%   0.97  
Gilles Simon             442     155    35.1%   0.96  
Richard Gasquet          370     159    43.0%   0.95

Isner still stands at the top of the leaderboard, while Bernard Tomic and Grigor Dimitrov give us a mild surprise by filling out the top three. Again, as the sample size increases, the variation decreases even further, illustrating that, over the long term, players tend to serve about as well at one score as they do at any other.

Forecasting the Effects of Performance Byes in Beijing

To the uninitiated, the WTA draw in Beijing this week looks a little strange. The 64-player draw includes four byes, which were given to the four semifinalists from last week’s event in Wuhan. So instead of empty places in the bracket next to the top four seeds, those free passes go to the 5th, 10th, and 15th seeds, along with one unseeded player, Venus Williams.

“Performance byes”–those given to players based on their results the previous week, rather than their seed–have occasionally featured in WTA draws over the last few years. If you’re interested in their recent history, Victoria Chiesa wrote an excellent overview.

I’m interested in measuring the benefit these byes confer on the recipients–and the negative effect they have on the players who would have received those byes had they been awarded in the usual way. I’ve written about the effects of byes before, but I haven’t contrasted different approaches to awarding them.

This week, the beneficiaries are Garbine Muguruza, Angelique Kerber, Roberta Vinci, and Venus Williams. The top four seeds–the women who were atypically required to play first-round matches, were Simona Halep, Petra Kvitova, Flavia Pennetta, and Agnieszka Radwanska.

To quantify the impact of the various possible formats of a 64-player draw, I used a variety of tools: Elo to rate players and predict match outcomes, Monte Carlo tournament simulations to consider many different permutations of each draw, and a modified version of my code to “reseed” brackets. While this is complicated stuff under the hood, the results aren’t that opaque.

Here are three different types of 64-player draws that Beijing might have employed:

  1. Performance byes to last week’s semifinalists. This gives a substantial boost to the players receiving byes, and compared to any other format, has a negative effect on top players. Not only are the top four seeds required to play a first-round match, they are a bit more likely to play last week’s semifinalists, since the byes give those players a better chance of advancing.
  2. Byes to the top four seeds. The top four seeds get an obvious boost, and everyone else suffers a bit, as they are that much more likely to face the top four.
  3. No byes: 64 players in the draw instead of 60. The clear winners in this scenario are the players who wouldn’t otherwise make it into the main draw. Unseeded players (excluding Venus) also benefit slightly, as the lack of byes mean that top players are less likely to advance.

Let’s crunch the numbers. For each of the three scenarios, I ran simulations based on the field without knowing how the draw turned out. That is, Kvitova is always seeded second, but she doesn’t always play Sara Errani in the first round. This approach eliminates any biases in the actual draw. To simulate the 64-player field, I added the four top-ranked players who lost in the final round of qualifying.

To compare the effects of each draw type on every player, I calculated “expected points” based on their probability of reaching each round. For instance, if Halep entered the tournament with a 20% chance of winning the event with its 1,000 ranking points, she’d have 200 “expected points,” plus her expected points for the higher probabilities (and lower number of points) of reaching every round in between. It’s simply a way of combining a lot of probabilities into a single easier-to-understand number.

Here are the expected points in each draw scenario (plus the actual Beijing draw) for the top four players, the four players who received performance byes, plus a couple of others (Belinda Bencic and Caroline Wozniacki) who rated particularly highly:

Player               Seed  PerfByes  TopByes  NoByes  Actual  
Simona Halep            1       323      364     330     341  
Petra Kvitova           2       276      323     290     291  
Venus Williams                  247      216     218     279  
Belinda Bencic         11       255      249     268     254  
Garbine Muguruza        5       243      202     210     227  
Angelique Kerber       10       260      224     235     227  
Caroline Wozniacki      8       208      203     205     199  
Flavia Pennetta         3       142      177     144     195  
Agnieszka Radwanska     4       185      233     192     188  
Roberta Vinci          15       120       91      94      90

As expected, the top four seeds are expected to reap far more points when given first-round byes. It’s most noticeable for Pennetta and Radwanska, who would enjoy a 20% boost in expected points if given a first-round bye. Oddly, though, the draw worked out very favorably for Flavia–Elo gave her a 95% chance of beating her first-round opponent Xinyun Han, and her draw steered her relatively clear of other dangerous players in subsequent rounds.

Similarly, the performance byes are worth a 15 to 30% advantage in expected points to the players who receive them. Vinci is the biggest winner here, as we would generally expect from the player most likely to suffer an upset without the bye.

Like Pennetta, Venus was treated very well by the way the draw turned out. The bye already gave her an approximately 15% boost compared to her expectations without a bye, and the draw tacked another 13% onto that. Both the structure of the draw and some luck on draw day made her the event’s third most likely champion, while the other scenarios would have left her in fifth.

All byes–conventional or unconventional–work to the advantage of some players and against others. However they are granted, they tend to work in favor of those who are already successful, whether that success is over the course of a year or a single week.

Performance byes are easy enough to defend: They give successful players a bit more rest between two demanding events, and from the tour’s perspective, they make it a little more likely that last week’s best players won’t pull off of this week’s tourney. And if all byes tend to the make the rich a little richer, at least performance byes open the possibility of benefiting different players than usual.

The Slow but Steady Erosion of the Server’s Advantage

After a couple of weeks of data-driven skepticism, I can finally confirm a bit of tennis’s conventional wisdom. Over the course of a typical match, breaks of serve are a little easier to come by.

This result–based on tens of thousands of matches from the last few years–is similar for both men and women. After about twelve games (total, not service games for each player), a hold is roughly 2% less likely than it was in the first few games of the match. By the 25th game, a hold is approximately 5% less likely than at the beginning of the match.

To control for the vagaries of surface, opponent, and other conditions, I’ve compared each service game to the server’s hold percentage within that match. Only the closest matches are likely to go very long, so it’s important to compare the last games of those matches to games with similarly even opponents.

It seems that this effect is the result of one or both of two factors: server fatigue (which may have more of an effect on results than an equivalent amount of returner fatigue), and the returner’s increasing familiarity with the server. It would be difficult to separate these two–and with this dataset, probably impossible–so for today, let’s stick with the nature of the effect, not its causes.

The following graph shows the relative probability of a hold of serve based on how much of the match (in games) has been played:

Relative hold percentage

I’ve set the hold probability of the first game at 100%, so all other numbers are relative to that. I’ve excluded tiebreaks from these calculations, though I considered them when counting games–that is, the first game of the second set after a tiebreak is considered the 14th game, not the 13th.

The results get a lot noisier starting around the women’s 25th game and the men’s 35th game, for the simple reason that most matches don’t get that far. For example, while the WTA calculations are based on 11,000 matches, only one-third reached the 25th game and less than one-tenth made it to the 31st.

The general downward trend indicates that the fatigue and/or familiarity effect dwarfs the effect of new balls. I have found that in men’s matches, the age of balls has a very small effect on hold percentage, and in women’s matches, it has no effect. In any case, the steady ebb of the server’s advantage is a stronger effect.

It is likely that some players suffer more from fatigue or familiarity than others. Due to the smaller size of the per-player samples, especially beyond the 20th game or so, I’m reluctant to draw any strong conclusions. Still, there are some intriguing numbers for the players for whom the dataset contains the most matches.

Here, I’ve calculated the hold percentage for several top players at various stages of the match, relative to their hold percentage in the first ten games. Thus, a number below 100% indicates less frequent holds, while a number above 100% means more frequent holds:

Player                 Matches  11 to 20  21 to 30  31 to 50  
Tomas Berdych              337     98.5%     98.3%    101.5%  
David Ferrer               330     97.0%     99.4%    102.4%  
Novak Djokovic             325    100.1%    101.8%    101.7%  
Roger Federer              325    100.2%     99.6%    100.4%  
Andy Murray                295     97.7%     98.7%     97.9%  
Rafael Nadal               293     99.2%    100.3%     93.7%  
Jo-Wilfried Tsonga         255    100.4%    100.9%     99.6%  
Philipp Kohlschreiber      252    101.4%     97.9%     96.7%  
John Isner                 251    100.4%    100.4%    100.3%  
Player                 Matches  11 to 20  21 to 30  31 to 50  
Kevin Anderson             247    100.0%     98.1%     97.5%  
Richard Gasquet            246     99.1%     98.4%    105.1%  
Gilles Simon               245    100.1%    103.7%     95.0%  
Milos Raonic               238     97.1%     96.1%     96.7%  
Marin Cilic                238     95.4%     97.5%     94.5%  
Fabio Fognini              235    100.4%     99.6%     98.2%  
Kei Nishikori              233    101.8%    104.1%    107.2%  
Grigor Dimitrov            224    100.9%    100.3%     94.6%  
Andreas Seppi              221    106.4%    100.4%    103.1%  
Feliciano Lopez            221     99.2%     99.7%     98.4%  
Total                    23326     98.1%     96.1%     95.1%

While John Isner is steady throughout the stages of the match, other big servers such as Milos Raonic and Marin Cilic are less dominant as the match progresses. The players whose hold percentage improves through the match–such as Novak Djokovic and David Ferrer–tend to be those without big serves, so we may be looking at more of an overall fatigue effect in those cases.

The most extreme number in the table is Rafael Nadal‘s relative hold percentage after the 30th game. Perhaps after that much time on court, his opponents finally figure out how to defend against the ad-court slider.

Here are the same calculations for top WTA players:

Player                Matches  11 to 15  16 to 20  21 to 40  
Agnieszka Radwanska       299    101.0%    104.9%     98.0%  
Sara Errani               279     97.7%     91.2%     92.7%  
Caroline Wozniacki        279    103.1%    102.3%    104.9%  
Serena Williams           266    102.8%    102.4%    104.9%  
Angelique Kerber          265    101.9%    103.0%    101.5%  
Samantha Stosur           253     99.2%    105.0%     97.6%  
Carla Suarez Navarro      252    102.2%    101.8%     93.7%  
Petra Kvitova             251     93.9%    100.4%     95.9%  
Roberta Vinci             250     94.2%     97.9%     95.4%  
Ana Ivanovic              241    100.8%    106.0%     95.2%  
Jelena Jankovic           241    102.2%    108.7%     96.4%  
Player                Matches  11 to 15  16 to 20  21 to 40  
Maria Sharapova           236    100.1%    105.9%    104.9%  
Victoria Azarenka         228    100.6%    103.7%     97.8%  
Lucie Safarova            227    102.7%    100.5%     94.4%  
Simona Halep              224     89.2%     95.3%    101.7%  
Dominika Cibulkova        210     98.7%     89.9%     99.9%  
Alize Cornet              210     96.2%    102.8%     96.4%  
Andrea Petkovic           194    101.5%    104.2%    107.5%  
Sloane Stephens           185     97.5%     90.1%     88.7%  
Sabine Lisicki            185     97.4%     97.5%     96.6%  
Ekaterina Makarova        185     96.6%    102.8%     92.8%  
Flavia Pennetta           180    105.1%     92.9%    103.9%  
Total                   22406     98.6%     97.2%     95.0%

Here is some confirmation that Serena Williams–at least on serve–gets better as the match progresses. Many of the other players with the strongest serve results late in matches are those known for fitness (like Caroline Wozniacki) or steeliness (Maria Sharapova).

Whether the root cause is fatigue or familiarity, most players are less effective on serve as the match progresses. With further research, I hope we’ll be able to better understand the cause and determine whether there are advantages to serving particularly well at certain stages of the match.

The Odds of Successfully Serving Out the Set

Serving for the set is hard … or so they say. Like other familiar tennis conceits, this one is ripe for confirmation bias. Every time we see a player struggle to serve out a set, we’re tempted to comment on the particular challenge he faces. If he doesn’t struggle, we ignore it or, even worse, remark on how he achieved such an unusual feat.

My findings–based on point-by-point data from tens of thousands of matches from the last few seasons–follow a familiar refrain: If there’s an effect, it’s very minor. For many players, and for some substantial subsets of matches, breaks of serve appear to be less likely at these purportedly high-pressure service games of 5-4, 5-3 and the like.

In ATP tour-level matches, holds are almost exactly as common when serving for the set as at other stages of the match. For each match in the dataset, I found each player’s hold percentage for the match. If serving for the set were more difficult than serving in other situations, we would find that those “average” hold percentages would be higher than players’ success rates when serving for the set.

That isn’t the case. Considering over 20,000 “serving-for-the-set” games, players held serve only 0.7% less often than expected–a difference that shows up only once every 143 attempts. The result is the same when we limit the sample to “close” situations, where the server has a one-break advantage.

Only a few players have demonstrated any notable success or lack thereof. Andy Murray holds about 6% more often when serving for the set than his average rate, making him one of only four players (in my pool of 99 players with 1,000 or more service games) to outperform his own average by more than 5%.

On the WTA tour, serving for the set appears to be a bit more difficult. On average, players successfully serve out a set 3.4% less often than their average success rate, a difference that would show up about once every 30 attempts. Seven of the 85 players with 1,000 service games in the dataset were at least 10% less successful in serving-for-the-set situations than their own standard.

Maria Sharapova stands out at the other end of the spectrum, holding serve 3% more often than her average when serving for the set, and 7% more frequently than average when serving for the set with a single-break advantage. She’s one of 30 players for whom I was able to analyze at least 100 single-break opportunities, and the only one of them to exceed expectations by more than 5%.

Given the size of the sample–nearly 20,000 serving-for-the-set attempts, with almost 12,000 of them single-break opportunities–it seems likely that this is a real effect, however small. Strangely, though, the overall finding is different at the lower levels of the women’s game.

For women’s ITF main draw matches, I was able to look at another 30,000 serving-for-the-set attempts, and in these, players were 2.4% more successful than their own average in the match. In close sets, where the server held a one-break edge, the server’s advantage was even greater: 3.5% better than in other games.

If anything, I would have expected players at lower levels to exhibit greater effects in line with the conventional wisdom. If it’s difficult to serve in high-pressure situations, it would make sense if lower-ranked players (who, presumably, have less experience with and/or are less adept in these situations) were not as effective. Yet the opposite appears to be true.

Lower-level averages from the men’s tour don’t shed much light, either. In main draw matches at Challengers, players hold 1.4% less often when serving for the set, and 1.8% less often with a single-break advantage. In futures main draws, they are exactly as successful when serving for the set as they are the rest of the time, regardless of their lead. In all of the samples, there are only a handful of players whose record is 10% better or worse when serving for the set, and a small percentage who over- or underperform by even 5%.

The more specific situations I analyze, the more the evidence piles up that games and points are, for the most part, independent–that is, players are roughly as effective at one score as they are at any other, and it doesn’t matter a great deal what sequence of points or games got them there. There are still plenty of situations that haven’t yet been analyzed, but if the ones that we talk about the most don’t exhibit the strong effects that we think they do, that casts quite a bit of doubt on the likelihood that we’ll find notable effects elsewhere.

If there is any truth to claims like those about the difficulty of serving for the set, perhaps it is the case that the pressure affects both players equally. After all, if a server needs to hold at 5-4, it is equally important for the returner to seize the final break opportunity. Maybe the level of both players drops, something we might be able to determine by analyzing how these points are played.

For now, though, we can conclude that players–regardless of gender or level–serve out the set about as often as they successfully hold at 1-2, or 3-3, or any other particular score.

The Pervasive Role of Luck in Tennis

No matter what the scale, from a single point to a season-long ranking–even to a career–luck plays a huge role in tennis. Sometimes good luck and bad luck cancel each other out, as is the case when two players benefit from net cord winners in the same match. But sometimes luck spawns more of the same, giving fortunate players opportunities that, in turn, make them more fortunate still.

Usually, we refer to luck only in passing, as one possible explanation for an isolated phenomenon. It’s important that we examine them in conjunction with each other to get a better sense of just how much of a factor luck can be.

Single points

Usually, we’re comfortable saying that the results of individual points are based on skill. Occasionally, though, something happens to give the point to an undeserving player. The most obvious examples are points heavily influenced by a net cord or a bad bounce off an uneven surface, but there are others.

Officiating gets in the way, too. A bad call that the chair umpire doesn’t overturn can hand a point to the wrong player. Even if the chair umpire (or Hawkeye) does overrule a bad call, it can result in the point being replayed–even if one player was completely in control of the point.

We can go a bit further into the territory of “lucky shots,” including successful mishits, or even highlight-reel tweeners that a player could never replicate. While the line between truly lucky shots and successful low-percentage shots is an ambiguous one, we should remember that in the most extreme cases, skill isn’t the only thing determining the outcome of the point.

Lucky matches

More than 5% of matches on the ATP tour this year have been won by a player who failed to win more than half of points played. Another 25% were won by a player who failed to win more than 53% of points–a range that doesn’t guarantee victory.

Depending on what you think about clutch and momentum in tennis, you might not view some–or even any–of those outcomes as lucky. If a player converts all five of his break point opportunities and wins a match despite only winning 49% of total points, perhaps he deserved it more. The same goes for strong performance in a tiebreaks, another cluster of high-leverage points that can swing a match away from the player who won more points.

But when the margins are so small that executing at just one or two key moments can flip the result–especially when we know that points are themselves influenced by luck–we have to view at least some of these tight matches as having lucky outcomes. We don’t have to decide which is which, we simply need to acknowledge that some matches aren’t won by the better player, even if we use the very loose definition of “better player that day.”

Longer-term luck

Perhaps the most obvious manifestation of luck in tennis is in the draw each week. An unseeded player might start his tournament with an unwinnable match against a top seed or with a cakewalk against a low-ranked wild card. Even seeded players can be affected by fortune, depending on which unseeded players they draw, along with which fellow seeds they will face at which points in the match.

Another form of long-term luck–which is itself affected by draw luck–is what we might call “clustering.” A player who goes 20-20 on a season by winning all of his first-round matches and losing all of his second-round matches will not fare nearly as well in terms of rankings or prize money as someone who goes 20-20 by winning only 10 first-round matches, but reaching the third round every time he does.

Again, this may not be entirely luck–this sort of player would quickly be labeled “streaky,” but combined with draw luck, he might simply be facing players he can beat in clusters, instead of getting easy first-rounders and difficult second-rounders.

The Matthew effect

All of these forms of tennis-playing fortune are in some way related. The sociologist Robert Merton coined the term “Matthew effect“–alternatively known as the principle of cumulative advantage–to refer to situations where one entity with a very small advantage will, by the very nature of a system, end up with a much larger advantage.

The Matthew effect applies to a wide range of phenomena, and I think it’s instructive here. Consider the case of two players separated by only a few points in the rankings–a margin that could have come about by pure luck: for instance, when one player won a match by walkover. One of these players gets the 32nd seed at the Australian Open and the other is unseeded.

These two players–who are virtually indistinguishable, remember–face very different challenges. One is guaranteed two matches against unseeded opponents, while the other will almost definitely face a seed before the third round, perhaps even a high seed in the first. The unseeded player might get lucky, either in his draw or in his matches, cancelling out the effect of the seeding, but it’s more likely that the seeded player will walk away from the tournament with more points, solidifying the higher ranking–that he didn’t earn in the first place.

Making and breaking careers

The Matthew effect can have an impact on an even broader scale. Today’s tennis pros have been training and competing from a young age, and most of them have gotten quite a bit of help along the way, whether it’s the right coach, support from a national federation, or well-timed wild cards.

It’s tough to quantify things like the effect of a good or bad coach at age 15, but wild cards are a more easily understood example of the phenomenon. The unlucky unseeded player I discussed above at least got to enter the tournament. But when a Grand Slam-hosting federation decides which promising prospect gets a wild card, it’s all or nothing: One player gets a huge opportunity (cash and ranking points, even if they lose in the first round!) while the other one gets nothing.

This, in a nutshell, is why people like me spend so much time on our hobby horses ranting about wild cards. It isn’t the single tournament entry that’s the problem, it’s the cascading opportunities it can generate. Sure, sometimes it turns into nothing–Ryan Harrison’s career is starting to look that way–but even in those cases, we never hear about the players who didn’t get the wild cards, the ones who never had the chance to gain from the cumulative advantage of a small leg up.

Why all this luck matters

If you’re an avid tennis fan, most of this isn’t news to you. Sure, players face good and bad breaks, they get good and bad draws, and they’ve faced uneven challenges along the way.

By discussing all of these types of fortune in one place, I hope to emphasize just how much luck plays a part in our estimate of each player at any given time. It’s no accident that mid-range players bounce around the rankings so much. Some of them are truly streaky, and injuries play a part, but much of the variance can be explained by these varying forms of luck. The #30 player in the rankings is probably better than the #50 player, but it’s no guarantee. It doesn’t take much misfortune–especially when bad luck starts to breed more opportunities for bad luck–to tumble down the list.

Even if many of the forms of luck I’ve discussed are truly skill-based and, say, break point conversions are a matter of someone playing better that day, the evidence generally shows that major rises and falls in things like tiebreak winning percentage and break point conversion rates are temporary–they don’t persist from year to year. That may not be properly classed as luck, but if we’re projecting the rankings a year from now, it might as well be.

While match results, tournament outcomes, and the weekly rankings are written in stone, the way that players get there is not nearly so clear. We’d do well to accept that uncertainty.

How Important is the Seventh Game of the Set?

Few nuggets of tennis’s conventional wisdom are more standard than the notion that the seventh game of each set is particularly crucial. While it’s often difficult to pin down such a well-worn conceit, it seems to combine two separate beliefs:

  1. If a set has reached 3-3, the pressure is starting to mount, and the server is less likely to hold serve.
  2. The seventh game is somehow more important than its immediate effect on the score, perhaps because the winner gains momentum by taking such a pivotal game.

Let’s test both.

Holding at 3-3

Drawing on my database of over 11,000 ATP tour-level matches from the last few years, I found 11,421 sets that reached three-all. For each, I calculated the theoretical likelihood that the server would hold (based on his rate of service points won throughout the match) and his percentage of service games won in the match. If the conventional wisdom is true, the percentage of games won by the server at 3-3 should be noticeably lower.

It isn’t. Using the theoretical model, these servers should have held 80.5% of the time. Based on their success holding serve throughout these matches, they should have held 80.2% of the time. At three-all, they held serve 79.5% of the time. That’s lower, but not enough lower that a human would ever notice. The difference between 80.2% and 79.5% is roughly one extra break at 3-3 per Grand Slam. Not Grand Slam match–an entire tournament.

None of that 0.7% discrepancy can be explained by the effect of old balls [1]. Because new balls are introduced after the first seven games of each match, the server at three-all in the first set is always using old balls, which should, according to another bit of conventional wisdom, work against him. However, the difference between actual holds and predicted holds at 3-3 is slightly greater after the first set: 78.9% instead of the predicted 79.8%. Still, this difference is not enough to merit the weight we give to the seventh game.

The simple part of our work is done: Servers hold at three-all almost as often as they do at any other stage of a match.

Momentum from the seventh game

At 3-3, a set is close, and every game matters. This is especially true in men’s tennis, where breaks are hard to come by. Against many players, getting broken so late in the set is almost the same as losing the set.

However, the focus on the seventh game is a bit odd. It’s important, but not as important as serving at 3-4, or 4-4, or 4-5, or … you get the idea. The closer a game to the end of the set, the more important it is–theoretically, anyway. If 3-3 is really worth the hoopla, it must grant the winner some additional momentum.

To measure the effect of the seventh game, I took another look at that pool of 11,000-plus sets that reached three-all. For each set, I calculated the two probabilities–based on each player’s service points won throughout the match–that the server would win the set:

  1. the 3-3 server’s chance of winning the set before the 3-3 game
  2. his chance of winning the set after winning or losing the 3-3 game

In this sample of matches, the average server at three-all had a 48.1% chance of winning the set before the seventh game. The servers went on to win 49.4% of the sets [2].

In over 9,000 of our 3-3 sets, the server held at 3-3. These players had, on average, a 51.3% chance of winning the set before serving at 3-3, which rose to an average of a 57.3% chance after holding. In fact, they won the set 58.6% of the time.

In the other 2,300 of our sets, the server failed to hold. Before serving at three-all, these players had a 35.9% chance of winning the set, which fell to 12.6% after losing serve. These players went on to win the set 13.7% of the time. In all of these cases, the model slightly underestimates the likelihood that the server at 3-3 goes on to win the set.

There’s no evidence here for momentum. Players who hold serve at three-all are slightly more likely to win the set than the model predicts, but the difference is no greater than that between the model and reality before the 3-3 game. In any event, the difference is small, affecting barely one set in one hundred.

When a server is broken at three-all, the evidence directly contradicts the momentum hypothesis. Yes, the server is much less likely to win the set–but that’s because he just got broken! The same would be true if we studied servers at 3-4, 4-4, 4-5, or 5-5. Once we factor in the mathematical implications of getting broken in the seventh game, servers are slightly more likely to win the set than the model suggests. Certainly the break does not swing any momentum in the direction of the successful returner.

There you have it. Players hold serve about as often as usual at three-all (whether they’re serving with new balls or not), and winning or losing the seventh game doesn’t have any discernible momentum effect on the rest of the set [3]. Be sure to tell your friendly neighborhood tennis pundits.

Continue reading How Important is the Seventh Game of the Set?

The Match Charting Project: Quick Start Guide

You’ve heard about the Match Charting Project, you’ve seen the amazingly detailed stats it generates, and you’ve decided it’s time to contribute. Here’s the simplest way to get started.

1. Choose a match. Check the list of charted matches (by date, or by player) and the Google doc of matches in progress. Once you’ve decided to chart a match, feel free to add yourself (along with the match) to the Google doc so that no one else will work on the same one.

Try to choose a relatively short match, and unless you really like Rafa, I’d suggest you avoid lefties for your first couple of attempts. It makes things a lot easier.

You can find full matches in many ways. There are plenty (though few very recent ones) on YouTube, many more on Asian video sites such as Soku, Daum, and Mgoon, and lots more if you have access to something like ESPN 3, TennisTV, or Tennis Channel Plus. There are also hundreds of archived ATP Challenger matches.

TennisTV and TC Plus are great because their players have buttons to skip forward or backward 10 seconds. Another alternative is to download videos to your local machine and then use a media player like SMPlayer or VLC, which allow you to move forward and backward through the match with quick keyboard shortcuts. Of course, DVRs work great for this, too.

2. Download the Match Charting Project spreadsheet and read through the “Instructions” tab. Charting a match involves a lot of details, but try not to get too bogged down. The most important things for beginners are:

  • serve direction (4 = wide, 5 = body, 6 = down the t)
  • the most common shot codes (f = forehand, b = backhand, s = backhand slice, r = forehand slice)
  • codes to indicate how the point ended (@ = unforced error, # = forced error, and * = winner)
  • codes to indicate the type of error (n = net, w = wide, d = deep, x = wide and deep).

The instructions cover several optional parts of the charting process, like shot direction and return depth. Including those makes things a lot more difficult, so for your first match, ignore them!

3. Start climbing the learning curve. I won’t deny it: It can be a bit frustrating to get started. The codes are a lot to remember, but trust me, it gets easier, especially if you stick to the basics. Many points look something like this:


That means: serve out wide, forehand return, forehand, backhand, backhand, forehand winner. That’s all!

It gets more complex when players approach the net or use less common tactics like dropshots. For your first match or two, you’ll probably consult the instructions frequently. Here’s another sample point:


Translated: Serve down the t (6), slice return (s), forehand volley (v), lob (l), overhead/smash (o) into the net (n) for an unforced error (@).

4. Be patient! After a few dozen points, you’ll start to get the hang of it. There will be plenty of rewinding, re-watching, and checking the instructions, but it will get considerably faster.

That’s it!

Once you’ve finished charting every point of the match, send me the spreadsheet and I’ll add it to the database.

After a match or two…

Of course, more data is more valuable, so once you’ve gotten the hang of the basics, it’s time to track more details of the match. But again–don’t rush into this! Adding these additional levels of complexity before you’re comfortable with the above will be very frustrating.

5. Shot direction. For every shot after the serve, use the number 1, 2, or 3 to indicate direction. 1 = to a right-hander’s forehand (or a lefty’s backhand), 2 = down the middle, or 3 = to a right-hander’s backhand. For example:


Translated: Serve to the body (5), forehand return down the middle (f2), forehand to (a righty’s) backhand side (f3), backhand crosscourt (b3), backhand down the line (b1) that missed wide (w) for a forced error (#).

When you’re comfortable with that:

6. Return depth. For service returns only, use an additional numeral for depth. 9 = very deep (the backmost quarter of the court), 8 = moderately deep (the next quarter, still behind the service line), and 7 = shallow (in the service boxes). For instance:


Meaning: Serve down the T (6), shallow slice return to (a righty’s) forehand side (s17), cross-court forehand winner (f1*).

Again, I have to ask you be patient with return depth: It’s the hardest step to add. In a very short period of time, you need to note the serve direction, return shot type, return direction, and return depth. It takes a bit of practice, but I’m convinced that recording return depth is worth it.

Finally, when you’re comfortable with all that, there’s one more thing to add:

7. Court position. A few symbols are used to record where players were when they hit certain shots. Most of the time they aren’t needed — a volley is almost always hit at net, while a backhand is almost always hit from the baseline. Use these codes for exceptions only:

  • The plus sign (+) is used for approach shots, including serves when a player serve-and-volleys.
  • The dash (-) indicates that a shot is hit at the net. Again, you don’t need to use it for “obvious” net shots like volleys, half-volleys, and smashes. It’s also unnecessary for the shot after a dropshot.
  • The (=) indicates that the shot was hit at the baseline. This is the least common, and usually is used for smashes hit from the baseline.

A couple more examples:


Translated: Server came in behind a serve out wide (4+), moderately deep slice return down the middle (s28), volley to (a righty’s) forehand side (v1), forehand winner hit from near the net (f-3*).

One more, which is just about as messy as it gets:


Meaning: Body serve (5), shallow forehand slice/chip return to (a righty’s) backhand side (r37), backhand crosscourt approach shot (b+3), backhand lob down the middle (m2), forehand lob to (a righty’s) forehand side (l1), crosscourt overhead/smash from the baseline (o=1), forehand slice/chip forced error (r#).

Happy charting! If you have any questions, please email me.