Is Grand Slam Qualifying Worth Tanking For?

Italian translation at settesei.it

Earlier today in Hobart, Naomi Osaka lost her second-round match to Mona Barthel. Coming into the match, she was in a tricky position: If she won, she wouldn’t be able to play Australian Open qualifying. For a young player outside the top 100, a tour-level quarterfinal would be nice, but presumably Melbourne was intended to be the centerpiece of her trip to Australia.

Since she lost the match, she’ll be able to play qualifying. But what if she hadn’t? Is this a situation in which a player would benefit from losing a match?

Put another way: In a position like Osaka’s, what are the incentives? If she could choose between the International-level quarterfinal and the Slam qualifying berth, which should she pick? Or, put more crassly, should a player in this position tank?

Let’s review the scenarios. In scenario A, Osaka wins the Hobart second-rounder, reaches the quarterfinal, and has a chance to go even further. She can’t play the Australian Open in any form. In scenario B, she loses the second-rounder, enters Melbourne qualifying and has a chance to reach the main draw.

Before we go through the numbers, take a guess: Which scenario is likely to give Osaka more ranking points? What about prize money?

Scenario A is more straightforward. By reaching the quarterfinals, she earns 30 additional ranking points and US$2,590 beyond what a second-round loser makes. Beyond that, we need to calculate “expected” points and prize money, using the amounts on offer for each round and combining them with her odds of getting there.

Let’s estimate that Osaka would have about a 25% chance of winning her quarterfinal match and earning an additional 50 points and $5400. In expected terms, that’s 12.5 points and $1,350. If she progresses, we’ll give her a 25% chance of reaching the final, then in the final, a 15% chance of winning the title.

Adding up these various possibilities, from her guaranteed QF points to her 0.94% chance (25%*25%*15*) of winning the Hobart title, we see that her expected rewards in scenario A are roughly 48 ranking points and just under $4,800.

Scenario B starts in a very different place. Thanks to the recent increases in Grand Slam prize money, every player in the qualifying takes home at least US$3,150. That’s already close to Osaka’s expected financial reward from advancing in Hobart. The points are a different story, though: First-round qualifying losers only get 2 WTA ranking points.

I’ll spare you all the calculations for scenario B, but I’ve assumed that Osaka would have a 70% chance of winning qualifying round 1, a 60% chance of winning QR2, and a 50% chance of winning QR3 and qualifying. Those might be a little bit high, but if they are, consider it compensation for the possibility that she’ll reach the main draw as a lucky loser. (Also, if we knock her chances all the way down to 50%, 45%, and 40%, the conclusions are the same, even if the points and prize money in scenario B are quite a bit lower.)

Those estimated probabilities translate into an expectation of about 23 ranking points and US$11,100. Osaka isn’t guaranteed any money beyond the initial $3,150, but the rewards for qualifying are enormous, especially compared to the prize money in Hobart. A first-round main draw loser in Melbourne takes home more money than the losing finalist does in Hobart.

And, of course, if she does qualify, there’s a chance she’ll go further. Since 2000, female Slam qualifiers have reached the second round 41% of the time, the third round 9% of the time, the fourth round 1.8% of the time, and the quarterfinals 0.3% of the time. Those odds, combined with her 21% chance of reaching the main draw in the first place,  translate into an additional 7 expected ranking points and $2,600 in prize money.

All told, scenario B gives us 30 expected ranking points and US$13,600 in expected prize money.

The Slam option results in far more cash, while the International route is worth more ranking points. In the long term, those ranking points would have some financial value, possible earning Osaka entry into a few higher-level events than she would otherwise qualify for. But that value probably doesn’t overcome the nearly $9,000 gap in immediate prize money.

I hope that no player ever tanks a match at a tour-level event so they can make it in time for Slam qualifying. But if one does, we’ll at least understand the logic behind it.

Winners, Errors, and Misinformation

Italian translation at settesei.it

Of the general ways in which points end–winners, unforced errors, and forced errors, which is the most common? It’s so basic a question that I’d never thought to investigate it. As it turns out, other people have, and they’re making tenuous claims based on their results.

A friend sent me a link to this advertisement for an instructional course, which–eventually, far into a painfully slow video–explains that more points on the pro tour end in forced errors than in winners or unforced errors. And because of this, the video argues, you can use some of the same patterns the pros use with the goal of generating forced errors. Apparently, aiming for winners is too risky, as is waiting for unforced errors.

Pedagogically, it seems reasonable enough to encourage patience and tactical conservatism. I don’t know the first thing about helping amateurs improve their tennis game, and I’ll happily defer to the experts.

However, the use of pro tennis data sparked my interest. I was immediately skeptical of these claims, which were apparently based on Grand Slam matches from 2012.

Using my datasets extracted from IBM Pointstream’s records of the last several slams, I tested the 2015 French Open and the 2015 US Open, tallying winners, unforced errors, and forced errors for men and women at both events. Here’s how they break down:

Dataset    Winners  Unforced  Forced  
FO Men       33.8%     32.9%   33.3%  
FO Women     32.7%     37.8%   29.5%  
                                      
USO Men      34.3%     31.6%   34.1%  
USO Women    31.0%     38.0%   30.9%

On both surfaces, men’s points split fairly evenly among the three categories. For women, winners are roughly even with forced errors (though there are more winners on clay) and unforced errors are the most common type of point-ending shot.

The Pointstream-based dataset has limitations, though, and you might have already guessed what it is. A sizable percentage of forced errors are serve returns, which don’t really seem pertinent to a discussion of tactics. We can separate aces from winners and double faults from unforced errors, but not forced error returns from forced errors.

For that, we need the resources of the Match Charting Project. That data gives us almost 1500 matches (evenly split between men and women) once we limit our view to tour-level contests. The MCP dataset contains everything Pointstream does–winners, unforced and forced errors–and much, much more. For our purposes, the key addition is rally length, which allows to differentiate between forced error returns and forced errors that came later in rallies.

With the MCP data, we can remove serve statistics from this discussion altogether, excluding aces, double faults, and forced error returns, none of which are tactics in the sense we usually use the word.

Here’s the frequency of each type of point-ender:

Dataset  Winners  Unforced  Forced  
Men        32.5%     45.8%   21.7%  
Women      32.4%     49.4%   18.2%

When serves are no longer cluttering the picture, winners retain their relative importance, but the distribution of errors changes enormously. Now, we see that once the returner gets the ball back in play (or receives a serve he or she should be able to put back in play), unforced errors outnumber forced errors by more than two to one.

(I also calculated clay-specific numbers, and all the rates were within one percentage point of the overall averages.)

Forced errors are the most common type of point-ender in only 14 of 728 charted men’s matches and 4 of 751 charted women’s matches. Even if you’re concerned about the representativeness of the MCP sample or the error-labeling tendencies of the charters and add make substantial adjustments to allow for them, these results overwhelming establish that unforced errors are the most common way in which rallies end.

I’m not sure how applicable the tactics and tendencies of pro players are to amateur coaching, so it’s possible that these numbers are irrelevant to a great deal of coaching pedagogy. But if you’re going to base your instructional technique on pro tennis stats, it seems reasonable to start by getting the numbers right.

—

The Match Charting Project is making it possible to answer questions about tennis that were previously unanswerable. Project data is open to all researchers. Please help us grow the project by watching tennis and charting matches!

Winning Return Points When It Matters

In my post last week about players who have performed better than expected in tiebreaks (temporarily, anyway), I speculated that big servers may try harder in tiebreaks than in return games.

If we interpret “try harder” as “win points more frequently,” we can test it. With my point-by-point dataset, we can look at every top player in the men’s game and compare their return-point performance in tiebreaks to their return-point performance earlier in the set.

As it turns out, top players post better return numbers in tiebreaks than they do earlier in the set. I looked at every match in my dataset (most tour-level matches from the last few seasons) for the ATP top 50, and found that these players, on average, won 5.2% more return points than they did earlier in those sets.

That same group of players saw their serve performance decline slightly, by 1.1%. Since the top 50 frequently play each other, it’s no surprise that the serve and return numbers point in different directions. However, the return point increase and the serve point decrease don’t cancel each other out, suggesting that the top 50 is winning a particularly large number of tiebreaks against the rest of the pack, mostly by improving their return game once the tiebreak begins.

(There’s a little bit of confirmation bias here, since some of the players on the edge of the top 50 got there thanks to good luck in recent tiebreaks. However, most of top 50–especially those players who make up the largest part of this dataset–have been part of this sample of players for years, so the bias remains only minor.)

My initial speculation concerned big servers–the players who might reasonably relax during return games, knowing that they probably won’t break anyway. However, big servers aren’t any more likely than others to return better in tiebreaks. (Or, put another way, to return worse before tiebreaks.) John Isner, Ivo Karlovic, Kevin Anderson, and Roger Federer all win slightly more return points in tiebreaks than they do earlier in sets, but don’t improve as much as the 5.2% average. What’s more, Isner and Anderson improve their serve performance for tiebreaks slightly more than they do their return performance.

There are a few players who may be relaxing in return games. Bernard Tomic improves his return points won by a whopping 27% in tiebreaks, Marin Cilic improves by 16%, and Milos Raonic improves by 11%. Tomic and Raonic, in particular, are particularly ineffective in return games when they have a break advantage in the set (more on that in a moment), so it’s plausible they are saving their effort for more important moments.

Despite these examples, this is hardly a clear-cut phenomenon. Kei Nishikori, for example, ups his return game in tiebreaks almost as much as Cilic does, and we would never think of him as a big server, nor do I think he often shows signs of tactically relaxing in return games. We have plenty of data for most of these players, so many of these trends are more than just statistical noise, but the results for individual players don’t coalesce into any simple, overarching narratives about tiebreak tendencies.

There is one nearly universal tendency that turned up in this research. When leading a set by one break or more, almost every player returns worse. (Conversely, when down a break, almost every player serves better.) The typical top 50 player’s return game declines by almost 5%, meaning that a player winning 35% of return points falls to 33.4%.

Almost every player fits this pattern. 48 of the top 50–everyone except for David Ferrer and Aljaz Bedene–win fewer return points when up a break, and 46 of 50 win more service points when down a break.

Pinning down exactly why this is the case is–as usual–more difficult than establishing that the phenomenon exists. It may be that players are relaxing on return. A one-break advantage, especially late, is often enough to win the set, so it may make sense for players to conserve their energy for their own service games. Looking at it from the server’s perspective, that one-break disadvantage might remove some pressure.

What’s clear is this: Players return worse than usual when up a break, and better than usual in tiebreaks. The changes are much more pronounced for some ATPers than others, but there’s no clear relationship with big serving. As ever, tiebreaks remain fascinating and more than a little inscrutable.

The Luck of the Tiebreak, 2015 in Review

Tiebreak outcomes are influenced by luck a lot more than most people think. All else equal, big servers aren’t any more successful than weak servers, and one season’s tiebreak king is often the next season’s tiebreak chump.

I’ve written a lot about this in the past, so I won’t repeat myself too much. (If you want to read more, here’s a good place to start.) In short, the data shows this: Good players win more tiebreaks than bad players do, but only because they’re better in general, not because they have special tiebreak skills. Very few players perform better or worse than they usually do in tiebreaks.

In the past, I’ve found that three players–Roger Federer, Rafael Nadal, and John Isner–consistently increase their level in tiebreaks. In other words, when you calculate how many tiebreaks Federer (or Nadal, or Isner) should win based on his overall rate of serve and return points won, you discover than he wins even more tiebreaks than that.

In any given year, some players score very high or very low–winning or losing far more tiebreaks than their overall level of play would suggest that they should. But the vast majority of those players regress back to the mean in subsequent years.

Here’s a look at which players outperformed the most in 2015 (minimum 20 tiebreaks). TBExp is the number of tiebreaks we would expect them to win, given their usual rate of serve and return points won. TBOE (Tie Breaks Over Expectations) is the difference between the number they won and the number we’d expect them to win, and TBOR is that difference divided by total tiebreaks.

Player              TBs  TBWon  TBExp  TBOE   TBOR  
Stan Wawrinka        46     34   24.9   9.1  19.8%  
Martin Klizan        25     17   12.2   4.8  19.0%  
Marin Cilic          35     26   21.0   5.0  14.2%  
Tomas Berdych        34     24   20.0   4.0  11.7%  
John Isner           64     39   31.7   7.3  11.3%  
Feliciano Lopez      42     27   22.4   4.6  11.0%  
Jiri Vesely          28     16   13.2   2.8  10.1%  
Sam Groth            31     18   14.9   3.1  10.1%  
Gilles Muller        45     27   22.7   4.3   9.5%  
Gael Monfils         28     18   15.4   2.6   9.4%

There are a lot of big servers here (more on that later) and a lot of new faces. Federer and Nadal were roughly neutral in 2015, winning exactly as many tiebreaks as we’d expect. Of the tiebreak masters, only Isner remained among the leaders. He has never posted a season below +5% TBOR, and only twice has he been below +11% TBOR. Just from this leaderboard, you can tell how elite that is.

Along with Isner, we have Marin Cilic, Feliciano Lopez, Sam Groth, and Gilles Muller, all players one would reasonably consider to be big servers. As I mentioned above, big serving doesn’t typically correlate with exceeding tiebreak expectations. It may just be a fluke: Lopez was roughly neutral in 2013 and 2014, and -15% in 2012; Groth doesn’t have much of a tour-level track record, but was -5% in 2014; Muller has been up and down throughout his career; and Cilic almost always underperformed until 2013.

Adding to the “fluke” argument is the case of Ivo Karlovic. His -14% TBOR this year was one of the worst among players who contested 20 or more tiebreaks, and he’s been exactly neutral over the last decade.

Let’s take a closer look at a few players.

Stan Wawrinka: For the second year in a row, he won at least 15% more tiebreaks than expected. Whether it’s clutch, focus, or dumb luck, the shift in his tiebreak fortunes dovetails nicely with his upward career trajectory. From 2006-13, he only posted one season at neutral or better, and his overall TBOR of -9% was one of the worst in the game for that span.

Cilic’s story is similar. Before 2013, he posted only one season above expectations. Since then, he’s won 19%, 16%, and 14% more tiebreaks than expected.

While only anecdotes, these two cases contradict an idea I’ve heard quite a bit, that players weaken in the clutch as they get older. The subject often comes up in the context of Karlovic’s tiebreak futility or Federer’s break point frustrations. It’s tough to prove one way or the other, in part because there’s no generally accepted measure of clutch in tennis. (If indeed there is any persistent clutch skill.) Using a measure like TBOR is dangerous, both because it is so noisy, and because of survivorship bias–players who get worse as they get older are more likely to fall in the rankings and play fewer tour matches as a result.

Another complicating factor is worthy of further study. To estimate how many tiebreaks a player should win, we need to take our expectation from somewhere. I’m using each player’s overall rates of serve and return points won. But if a player is trying harder in tiebreaks (assuming more effort translates into better results), we would expect that he would win more points in tiebreaks.

Isner has admitted to coasting on unimportant points, and for someone with his game style, a whole lot of return points can be classified as unimportant. Very generally speaking, the more one-dimensional the player, the more reason he has to take it easy during return games, and the more he does so, the more we would observe that he outperforms expectations in tiebreaks–simply because he sets expectations artificially low.

That might be an explanation for Isner’s consistent appearance on these leaderboards. And if we assume that players become more strategically sound as they age–or simply better at tactically conserving energy–we might have a reason why older players score higher in this metric.

Two more players worth mentioning are Milos Raonic and Kei Nishikori. They were 5th and 6th on the 2014 leaderboard, outperforming expectations by 15% and 14%, respectively. In 2015, Raonic fell to neutral, and Nishikori (in far fewer tiebreaks) dropped to -14%, nearly the bottom of the rankings. Taken together, it’s a good reminder of the volatility of these numbers. In Raonic’s case, it’s a warning that relying too much on winning tiebreaks (which, by extension, implies relying too little on one’s return game) is a poor recipe for long-term success.

Finally, some notes on the big four. Novak Djokovic and Andy Murray have never figured heavily in these discussions, both because they don’t play a ton of tiebreaks, and because they don’t persistently out- or underperform expectations. Federer and Nadal, however, were long among the best. Both have returned to the middle of the pack: Federer hasn’t posted a TBOR above 5% since 2011, and Nadal underperformed by 8.5% in 2014 before bouncing back to neutral last season.

Whatever tiebreak skill Roger and Rafa once had now eludes them. On the other hand, ten months of good tiebreak luck can happen to anyone, even a legend. If either player can recapture that tiebreak magic–even if it’s mere luck that allows them to do so–it might translate into a few more wins as they try to reclaim the top spot in the rankings.

A New Year For the Match Charting Project

The 2015 tennis season was an amazing one for the Match Charting Project. We added more than 1,000 new matches to the database, including 800 from the 2015 season alone. In about two and half years, the project has grown from little more than a half-baked idea to a tremendous resource for tennis researchers.

The Match Charting Project relies on volunteers to record details of every point of professional matches. Over 50 of you have taken the time to learn the method and chart at least one match, and some of you have gone way, way beyond that. Taken together, the results are outstanding.

In a sport where most data is hidden away by federations and sponsors, the Match Charting Project is one of the few bright spots for analysts. Anyone can use this data to research players, tendencies, and tactics. Anyone can contribute and help us learn more about the game.

We now have shot-by-shot data for over 1,600 matches, including sizable samples for most of the current ATP and WTA top 40. We have particularly large datasets for some top players, including the ATP big four and several WTA favorites. The database includes at least one match for every player in the ATP and WTA top 100, as well as detailed records of matches for many notable retired players.

We made huge progress last year, but I think we can do even better.

In 2015, we added 1,069 matches to the database, just under three per day. At the end of the day on December 31st, we had a total of 1,617 matches covered.

My goal for 2016 is to double that:  another 1,617 new matches in 2016, a rate of about four and a half per day. To accomplish that, we’ll need more of you to pitch in. Hopefully those of you who have contributed in the past will continue to do so. Charting 1,600 matches is no easy feat, but with enough of us working toward that goal, we’ll get there.

For my part, in addition to charting an unhealthy number of matches, I’ll continue to write about my findings from the MCP dataset, and I’ll be developing ways to make the data more accessible to fans. Keep an eye out for updates–other researchers are working on projects that should create even more interest in the Match Charting Project.

Want to find out more? Ready to contribute? Here’s a list of MCP-related resources to fill you in on all the details of the project:

11 Reasons to Contribute to the Match Charting Project

Italian translation at settesei.it

In the last two years, more than 50 dedicated tennis fans have charted over 1,500 matches for the Match Charting Project. The results are amazing–detailed shot-by-shot data, covering everything from serve location to return depth to mid-rally tactics for hundreds of current and retired players.

We’re just getting started. Up to this point, the project has relied very heavily on a small number of contributors. Five of us have charted at least 95 matches each. If we’re going to continue to grow the project and chart more matches, we need more people charting.

I hope you will be one of those people. If you’re already convinced, here’s the “quick start” guide to charting. Otherwise, here are some of the reasons you should contribute to the Match Charting Project:

1. Learn more about tennis. The comment I hear most frequently from first-time charters is that they’re stunned by how much detail they notice while charting a match. When forced to pay attention to every shot of every point, you’ll pick up on things you’d otherwise ignore.
2. Watch more intently. The default sports-watching mode for many of us is to put a match on in the background, half-heartedly do something else, and tune back in for highlight replays or important moments. There’s a ton of great tennis being played that doesn’t fit those categories, and if you’re charting the match, you’ll see all of it.
3. Discover new players. If you’re curious about a prospect, or you want to know about a player who just beat your fave, charting a couple of matches is a great way to learn more.
4. Discover new things about your favorite players. With the focus that comes from charting every shot of a match, you’re likely to spot new aspects of anyone’s game, even if you’ve been watching them play for years.
5. Improve your tennis game. You may not think of your game as ripe for a tactical overhaul, but by paying close attention to professional points, you’ll see tactics that will improve your own performance, even if you can’t execute them perfectly.
6. Make your own narrative. When you watch every shot, you tend to notice patterns you might otherwise miss. If you’re sick of the tired tropes trotted out during so many tennis broadcasts (experience beating youth, aggression overwhelming caution, etc.), you’ll have the data to determine for yourself what’s really going on.
7. Contribute to the analytics movement. While the state of tennis data is mediocre, why not help improve it? For many players, one or two more charted matches will substantially increase the publicly-accessible knowledge of their game. And in the aggregate, the more matches we have, the better we can use the data to learn more about the game.
8. Gain the moral high ground. In the tennis twitterverse, whining about the state of tennis data is standard fare. I don’t have a lot of sympathy for those who complain without doing anything about it. Next time you want to vent your feelings about certain tennis organizations and their stat-keeping efforts, wouldn’t it feel better to know that you’re part of the solution?
9. Learn how to get more out of the data. If you want to use Match Charting Project data for your own research, the best way to learn what the dataset contains (as well as its limitations) is to chart a few matches.
10. Recognize patterns for further study. Looking for a research topic? Chart a couple of matches, look for patterns, make a few notes, and if you don’t have ten potential topics written down, you’re not trying hard enough.
11. It’s fun! Ok, it’s a bit cumbersome to get started. Bear with it for a match, and you’ll find that charting can make watching tennis even more enjoyable.

In 2015 alone, we’ll add over 1,000 new matches to the charting database. I hope to significantly improve on that in 2016, but I’ll need more help. It doesn’t have to take much–one hundred volunteers charting one match a month would more than double this year’s output. Please contribute!

The Match Charting Project Hits 1,400!

Yesterday, the Match Charting Project hit another milestone: 1,400 matches!

For the last few months, we’ve been growing at the fastest pace yet, better than four new matches per day. We’re adding high-profile matches from the men’s and women’s tour, along with plenty of Challenger and historical matches, as well.

Recent milestones include 100 Rafael Nadal matches, 75 Novak Djokovic matches, 300 active ATPers, 40 Agnieszka Radwanska matches, and 50 Grand Slam finals.

Here’s the complete list, where you can find detailed shot-by-shot breakdowns of every one of these matches, sorted by player.

I’ve also updated the raw data, which is available here.

We’ll definitely reach 1,500 by the end of the year. We started 2015 with 548 matches, so if we keep up the current rate, we’ll cross the 1,000 mark for 2015 alone, including nearly 700 from the 2015 season. To those of you who have contributed: Thank you. You deserve the thanks of the entire tennis community.

If you haven’t contributed, now is a great time to start. Click here for my “quick start” guide to match charting. As we approach the offseason, it’s the perfect opportunity to dig up one of those old matches you’ve always meant to watch. Learn to chart, and you can make tennis analytics better at the same time.

All the Answers

Italian translation at settesei.it

At the end of Turing’s Cathedral, George Dyson suggests that while computers aren’t always able to usefully respond to our questions, they are able to generate a stunning, unprecedented array of answers–even if the corresponding questions have never been asked.

Think of a search engine: It has indexed every possible word and phrase, in many cases still waiting for the first user to search for it.

Tennis Abstract is no different. Using the menus on the left-hand side of Roger Federer’s page–even ignoring the filters for head-to-heads, tournaments, countries, matchstats, and custom settings like those for date and rank–you can run five trillion different queries. That’s twelve zeroes–and that’s just Federer. Judging by my traffic numbers, it will be a bit longer before all of those have been tried.

Every filter is there for a reason–an attempt to answer some meaningful question about a player. But the vast majority of those five trillion queries settle debates that no one in their right mind would ever have, like Roger’s 2010 hard-court Masters record when winning a set 6-1 against a player outside the top 10. (He was 2-0.)

The danger in having all these answers is that it can be tempting to pretend we were asking the questions–or worse, that we were asking the questions and suspected all along that the answers would turn out this way.

The Hawkeye data on tennis broadcasts is a great example. When a graphic shows us the trajectory of several serves, or the path of the ball over every shot of a rally, we’re looking at an enormous amount of raw data, more than most of us could comprehend if it weren’t presented against the familiar backdrop of a tennis court. Given all those answers, our first instinct is too often to seek evidence for something we were already pretty sure about–that Jack Sock’s topspin is doing the damage, or Rafael Nadal’s second serve is attackable.

It’s tough to argue with those kind of claims, especially when a high-tech graphic appears to serve as confirmation. But while those graphics (or those results of long-tail Tennis Abstract queries) are “answers,” they address only narrow questions, rarely proving the points we pretend they do.

These narrow answers are merely jumping-off points for meaningful questions. Instead of looking at a breakdown of Novak Djokovic’s backhands over the course of a match and declaring, “I knew it, his down-the-line backhand is the best in the game,” we should realize we’re looking at a small sample, devoid of context, and take the opportunity to ask, “Is his down-the-line backhand always this good?” or “How does his down-the-line backhand compare to others?” Or even, “How much does a down-the-line backhand increase a player’s odds of winning a rally?”

Unfortunately, the discussion usually stops before a meaningful question is ever asked. Even without publicly released Hawkeye data, we’re beginning to have the necessary data to research many of these questions.

As much as we love to complain about the dearth of tennis analytics, too many people draw conclusions from the pseudo-answers of fancy graphics. With more data available to us than ever before, it is a shame to mistake narrow, facile answers for broad, meaningful ones.

The Difficulty (and Importance) of Finding the Backhand

Italian translation at settesei.it

One disadvantage of some one-handed backhands is that they tend to sit up a little more when they’re hit crosscourt. That gives an opponent more time to prepare and, often, enough time to run around a crosscourt shot and hit a forehand, which opens up more tactical possibilities.

With the 700 men’s matches in the Match Charting Project database (please contribute!), we can start to quantify this disadvantage–if indeed it has a negative effect on one-handers. Once we’ve determined whether one-handers can find their opponents’ backhands, we can try to answer the more important question of how much it matters.

The scenario

Let’s take all baseline rallies between right-handers. Your opponent hits a shot to your backhand side, and you have three choices: drive (flat or topspin) backhand, slice backhand, or run around to hit a forehand. You’ll occasionally go for a winner down the line and you’ll sometimes be forced to hit a weak reply down the middle, but usually, your goal is to return the shot crosscourt, ideally finding your opponent’s backhand.

Considering all righty-righty matchups including at least one player among the last week’s ATP top 72 (I wanted to include Nicolas Almagro), here are the frequency and results of each of those choices:

SHOT    FREQ  FH REP  BH REP    UFE  WINNER  PT WON  
ALL             9.9%   68.1%  10.8%    5.8%   43.1%  
SLICE  11.9%   34.1%   49.5%   7.1%    0.6%   40.2%  
FH     44.9%    2.8%   69.0%  13.0%    9.8%   42.1%  
BH     43.3%   10.7%   72.2%   9.5%    3.1%   45.0%  
                                                     
1HBH   42.6%   12.0%   69.5%   9.3%    3.8%   44.2%  
2HBH   43.5%   10.0%   73.4%   9.6%    2.8%   45.4%

“FH REP” and “BH REP” refer to a forehand or backhand reply, and we can see just how much shot selection matters in keeping the ball away from your opponent’s forehand. A slice does a very poor job, while an inside-out forehand almost guarantees a backhand reply, though it comes with an increased risk of error.

The differences between one- and two-handed backhands aren’t as stark. One-handers don’t find the backhand quite as frequently, though they hit a few more winners. They hit drive backhands a bit less often, but that doesn’t necessarily mean they are hitting forehands instead. On average, two-handers hit a few more forehands from the backhand corner, while one-handers are forced to hit more slices.

One hand, many types

Not all one-handed backhands are created equal, and these numbers bear that out. Stanislas Wawrinka‘s backhand is as effective as the best two-handers, while Roger Federer‘s is typically the jumping-off point for discussions of why the one-hander is dying.

Here are the 28 players for whom we have at least 500 instances (excluding service returns) when the player responded to a shot hit to his backhand corner. For each, I’ve shown how often he chose a drive backhand or forehand, and the frequency with which he found the backhand–excluding his own errors and winners.

Player                 BH  BH FRQ  FIND BH%  FH FRQ  FIND BH%  
Alexandr Dolgopolov     2   45.7%     94.2%   43.3%     98.7%  
Kei Nishikori           2   51.1%     94.0%   38.9%     98.1%  
Andy Murray             2   41.0%     92.4%   46.5%     98.6%  
Stanislas Wawrinka      1   48.6%     92.1%   37.5%     98.0%  
Bernard Tomic           2   33.8%     91.7%   43.8%     97.9%  
Novak Djokovic          2   47.2%     91.7%   41.4%     98.5%  
Kevin Anderson          2   41.0%     91.5%   45.8%     96.6%  
Borna Coric             2   46.5%     90.7%   44.2%     96.9%  
Pablo Cuevas            1   41.9%     90.6%   54.5%     96.5%  
Marin Cilic             2   45.4%     89.7%   43.3%     97.2%  
                                                               
Player                 BH  BH FRQ  FIND BH%  FH FRQ  FIND BH%  
Tomas Berdych           2   41.6%     89.3%   44.2%     97.5%  
Pablo Carreno Busta     2   55.4%     87.8%   41.1%     93.5%  
Fabio Fognini           2   46.0%     87.4%   47.0%     96.1%  
Richard Gasquet         1   57.2%     87.3%   32.1%     96.8%  
Andreas Seppi           2   40.3%     87.2%   50.0%     93.9%  
Nicolas Almagro         1   53.6%     86.5%   39.3%     98.0%  
Dominic Thiem           1   38.5%     86.2%   50.0%     96.5%  
Gael Monfils            2   48.0%     85.3%   46.3%     85.3%  
David Ferrer            2   48.2%     84.9%   40.4%     97.1%  
Roger Federer           1   42.7%     84.8%   43.6%     94.5%  
                                                               
Player                 BH  BH FRQ  FIND BH%  FH FRQ  FIND BH%  
Gilles Simon            2   46.9%     84.6%   46.5%     94.6%  
David Goffin            2   45.4%     84.6%   45.7%     94.9%  
Roberto Bautista Agut   2   39.6%     83.3%   46.7%     98.4%  
Jo Wilfried Tsonga      2   43.5%     82.0%   44.5%     96.3%  
Grigor Dimitrov         1   41.4%     78.6%   39.4%     92.8%  
Milos Raonic            2   31.5%     63.5%   56.5%     94.3%  
Jack Sock               2   27.0%     62.5%   62.9%     96.3%  
Tommy Robredo           1   26.6%     56.1%   62.3%     88.4%

One-handers Wawrinka, Pablo Cuevas, and Richard Gasquet (barely) are among the top half of these players, in terms of finding the backhand with their own backhand. Federer and his would-be clone Grigor Dimitrov are at the other end of the spectrum.

Taking all 60 righties I included in this analysis (not just those shown above), there is a mild negative correlation (r^2 = -0.16) between a player’s likelihood of finding the opponent’s backhand with his own and the rate at which he chooses to hit a forehand from that corner. In other words, the worse he is at finding the backhand, the more inside-out forehands he hits. Tommy Robredo and Jack Sock are the one- and two-handed poster boys for this, struggling more than any other players to find the backhand, and compensating by hitting as many forehands as possible.

However, Federer–and, to an even greater extent, Dimitrov–don’t fit this mold. The average one-hander runs around balls in their backhand corner 44.6% of the time, while Fed is one percentage point under that and Dimitrov is below 40%. Federer is perceived to be particularly aggressive with his inside-out (and inside-in) forehands, but that may be because he chooses his moments wisely.

Ultimate outcomes

Let’s look at this from one more angle. In the end, what matters is whether you win the point, no matter how you get there. For each of the 28 players listed above, I calculated the rate at which they won points for each shot selection. For instance, when Novak Djokovic hits a drive backhand from his backhand corner, he wins the point 45.4% of the time, compared to 42.3% when he hits a slice and 42.4% when he hits a forehand.

Against his own average, Djokovic is about 3.6% better when he chooses (or to think of it another way, is able to choose) a drive backhand. For all of these players, here’s how each of the three shot choices compare to their average outcome:

Player                 BH   BH W   SL W   FH W  
Dominic Thiem           1  1.209  0.633  0.924  
David Goffin            2  1.111  0.656  0.956  
Grigor Dimitrov         1  1.104  0.730  1.022  
Gilles Simon            2  1.097  0.922  0.913  
Tomas Berdych           2  1.085  0.884  0.957  
Pablo Carreno Busta     2  1.081  0.982  0.892  
Kei Nishikori           2  1.070  0.777  0.965  
Roberto Bautista Agut   2  1.055  0.747  1.027  
Stanislas Wawrinka      1  1.050  0.995  0.936  
Borna Coric             2  1.049  1.033  0.941  
                                                
Player                 BH   BH W   SL W   FH W  
Bernard Tomic           2  1.049  1.037  0.943  
Jack Sock               2  1.049  0.811  1.010  
Gael Monfils            2  1.048  1.100  0.938  
Fabio Fognini           2  1.048  0.775  0.987  
Milos Raonic            2  1.048  0.996  0.974  
Nicolas Almagro         1  1.046  0.848  0.964  
Kevin Anderson          2  1.038  1.056  0.950  
Novak Djokovic          2  1.036  0.966  0.969  
Andy Murray             2  1.031  1.039  0.962  
Roger Federer           1  1.023  1.005  0.976  
                                                
Player                 BH   BH W   SL W   FH W  
Richard Gasquet         1  1.020  0.795  1.033  
Andreas Seppi           2  1.019  0.883  1.008  
David Ferrer            2  1.018  0.853  1.020  
Alexandr Dolgopolov     2  1.010  1.010  0.987  
Marin Cilic             2  1.006  1.009  0.991  
Pablo Cuevas            1  0.987  0.425  1.048  
Jo Wilfried Tsonga      2  0.956  0.805  1.095  
Tommy Robredo           1  0.845  0.930  1.079

In this view, Dimitrov–along with his fellow one-handed flame carrier Dominic Thiem–looks a lot better. His crosscourt backhand doesn’t find many backhands, but it is by far his most effective shot from his own backhand corner. We would expect him to win more points with a drive backhand than with a slice (since he probably opts for slices in more defensive positions), but it’s surprising to me that his backhand is so much better than the inside-out forehand.

While Dimitrov and Thiem are more extreme than most, almost all of these players have better results with crosscourt drive backhands than with inside-out (or inside-in forehands). Only five–including Robredo but, shockingly, not including Sock–win more points after hitting forehands from the backhand corner.

It’s clear that one-handers do, in fact, have a slightly more difficult time forcing their opponents to hit backhands. It’s much less clear how much it matters. Even Federer, with his famously dodgy backhand and even more famously dominant inside-out forehand, is slightly better off hitting a backhand from his backhand corner. We’ll never know what would happen if Fed had Djokovic’s backhand instead, but even though Federer’s one-hander isn’t finding as many backhands as Novak’s two-hander does, it’s getting the job done at a surprisingly high rate.

Toward a Better Understanding of Return Effectiveness

Italian translation at settesei.it

The deeper the return, the better, right? That, at least, is the basis for many of the flashy graphics we see these days on tennis broadcasts, indicating the location of every return, often separated into “shallow,” “medium,” and “deep” zones.

In general, yes, deep returns are better than shallow ones. But return winners aren’t overwhelmingly deep, since returners can achieve sharper angles if they aim closer to the service line. There are plenty of other complicating factors as well: returns to the sides of the court are more effective than those down the middle, second-serve returns tend to be better than first-serve returns, and topspin returns result in more return points won than chip or slice returns.

While most of this is common sense, quantifying it is an arduous and mind-bending task. When we consider all these variables–first or second serve, deuce or ad court, serve direction, whether the returner is a righty or lefty, forehand or backhand return, topspin or slice, return direction, and return depth–we end up with more than 8,500 permutations. Many are useless (righties don’t hit a lot of forehand chip returns against deuce court serves down the T), but thousands reflect some common-enough scenario.

To get us started, let’s set aside all of the variables but one. When we analyze 600+ ATP matches in the Match Charting Project data, we have roughly 61,000 in-play returns coded in one of nine zones, including at least 2,000 in each.  Here is a look at the impact of return location, showing the server’s winning percentage when a return comes back in play to one of the nine zones:rzones1show

(“Shallow” is defined as anywhere inside the service boxes, while “Medium” and “Deep” each represent half of the area behind the service boxes. The left, center, and right zones are intended to indicate roughly where the return would cross the baseline, so for sharply angled shots, a return might bounce shallow near the middle of the court but be classified as a return to the forehand or backhand side.)

As we would expect, deeper returns work in favor of the returner, as do returns away from the center of the court. A bit surprisingly, returns to the server’s forehand side (if he’s a right-hander) are markedly more effective than those to the backhand. This is probably because right-handed returners are most dangerous when hitting crosscourt forehands, although left-handed returners are also more effective (if not as dramatically) when returning to that side of the court.

Let’s narrow things down just a little and see how the impact of return location differs on first and second serves. Here are the server’s chances of winning the point if a first-serve return comes back in each of the nine zones:

rzones2showF

And the same for second-serve returns:

rzones3showF

It’s worth emphasizing just how much impact a deep return can have. So many points are won with unreturnable serves–even seconds–that simply getting the ball back in play comes close to making the point a 50/50 proposition. A deep second-serve return, especially to a corner, puts the returner in a very favorable position. Consistently hitting returns like that is a big reason why Novak Djokovic essentially turns his opponents’ second serves against them.

The final map makes it clear how valuable it is to move the server away from the middle of the court. Think of it as a tactical first strike, forcing the server to play defensively instead of dictating play with his second shot. Among second-serve returns put in play, any ball placed away from the middle of the court–regardless of depth–gives the returner a better chance of winning the point than does a deep return down the middle.

For today, I’m going to stop here. This is just the tip of the iceberg, as there are so many variables that play some part in the effectiveness of various service returns. Ultimately, understanding the potency of each return location will give us additional insight into what players can achieve with different kinds of serve, which players are deadliest with certain types of returns, and how best to handle different returns with the server’s crucial second shot.