Research – Page 2 – Heavy Topspin

The Underserved First Point

Not all points are created equal. Ask around, and you’ll get a variety of opinions as to which points are most important. Break points, obviously, are key. Pundits are fond of 15-30.

Then there’s the first point of the game. It’s been conventional wisdom for a long time that the opening points holds disproportionate weight. In a previous study, I disproved that. Of course it’s valuable to move from 0-0 to 15-0, and no one likes to start a game by dropping to 0-15. But the first point doesn’t have any magical effect on the outcome of the game beyond simply adding to one or the other player’s tally.

Yet here I am, talking about the first point again. While there still isn’t any magic, the first point is going to the returner too often. With a slight change in tactics or focus, this is a rare analytical insight that pros may be able to use to win a few more service games.

Point by point

The balance between the server and returner varies a great deal depending on the point score. In men’s singles matches at the US Open between 2019 and 2021, servers won 63.6% of points in non-tiebreak games. Yet at 40-love, the server won 67.7%, and at ad-out, the server won only 59.6%.

The point scores that generated such extremes hint at what’s going on here. If a game has reached 40-love, the server is probably a good one. It’s not always the case, but if you look at all the 40-love games in a large dataset, you’ll get far more John Isner holds than Benoit Paire holds. The opposite applies to ad-out, a score that Isner rarely faces. Thus, the difference in point-by-point serve percentage isn’t (entirely) because of the point score–it’s because of the servers who get there.

Other differences are more prosaic. On average, servers win more deuce-court points than ad-court points. In the same three-year dataset, the difference was 64.2% to 62.9%. There’s no selection bias component here. The typical ATPer is simply stronger in that direction. Some players–particularly left-handers–break the mold, but most will favor the deuce side. Both Novak Djokovic and Roger Federer, for instance, win nearly two percentage points more often when serving to that court.

Unbiasing

Because scores like 40-love and ad-out aren’t randomly distributed among servers, we need to do a bit more work to figure out which scores really do favor the server. The trick here is to compare each service point to the rest of the server’s points in the same match. A point like 40-love has a ton of Isners and Opelkas in it, so we’ll end up comparing it to a lot of other Isner and Opelka points. And in fact, the average player who reaches 40-love wins 65.0% of their service points and 64.3% in the ad court, two numbers that are well above average.

Working through the same exercise for every point score gives us a list of “actual” serve points won, “expected” serve points won, and differences. The “actual” column tells us what really happened at that score, bias and all; “expected” tells us how often that particular set of players won service points during the entire matches in question; and the difference gives us a first look at where servers are over- or under-performing.

The following table shows these numbers for each point score:

Score  Actual  Expected  Difference  
40-AD   59.6%     61.4%       -1.8%  
0-0     63.3%     64.6%       -1.3%  
15-0    62.7%     63.3%       -0.6%  
40-30   61.6%     62.2%       -0.6%  
15-30   62.3%     62.7%       -0.4%  
30-0    64.7%     65.1%       -0.3%  
40-40   62.6%     62.8%       -0.1%  
0-15    63.2%     63.3%       -0.1%  
                                     
Score  Actual  Expected  Difference  
40-15   64.6%     64.5%        0.0%  
30-15   62.8%     62.7%        0.1%  
AD-40   61.6%     61.4%        0.2%  
30-30   64.0%     63.6%        0.4%  
0-30    65.9%     65.2%        0.8%  
15-15   64.8%     64.0%        0.8%  
30-40   63.6%     62.2%        1.4%  
0-40    66.1%     64.7%        1.4%  
15-40   66.9%     64.5%        2.4%  
40-0    67.7%     64.3%        3.4%

The scores at the top of the table are the ones where we would expect servers to win more points. At the bottom of the list are those where the server seems to overperform.

Some of the results lend themselves to easy narratives. Servers really focus at 0-40 and 15-40, while returners know they have more break chances coming. 40-AD (ad-out) seems like a stressful time to serve, and the numbers back that up. Other results are a bit more baffling–shouldn’t 30-30 and 40-40 be the same, since they are logically equivalent? Why are servers performing so well at 30-40 if they ultimately struggle at 40-AD?

And to today’s topic: What about the first point? It ranks second only to 40-AD in how much the server underperforms, despite no obvious reason why it should lean one way or the other.

Second to none

When we consider a few more factors, this first-point underperformance has an even greater impact.

One useful way to measure the importance of a point is with win probability. Given any point score (or set/game/point score), combined with the likelihood that the server will win any given point, you can calculate the probability of a hold (or a match victory). If we assume that the server wins 64.2% of points, he’ll hold 81.6% of the time, so his win probability at the beginning of the game is 81.6%.

* 64.2% was the rate in non-tiebreak games at the 2021 US Open, while the overall rate for this 2019-21 dataset is a bit lower.

The next concept is volatility. A point’s volatility is determined by how much the result could swing the win probability. By winning the first point, the server’s win probability rises to 89.7%, the figure for such a server at 15-love. If he loses, it falls to 67.2%. The difference–22.5%–tells us how much is at stake in that single point.

In volatility terms, the first point isn’t particularly crucial. A 22.5% swing far outstrips, say, the 9.3% volatility at 30-love, but it pales next to the 76.3% volatility at 30-40. When the server faces break point, one swing of the racket can determine whether win probability drops to zero (because he loses the game), or bounces back north of 50% (because he gets back to deuce).

What the first point of the game gives up in volatility, it wins back in volume. The stakes are never higher than at 40-AD, but at the US Open in the last few years, barely one-fifth of games ever get that far. By contrast, there’s a love-love kickoff in every single game.

By combining volatility and volume with the degree to which servers under- or over-perform, we can put together a top-level view of what players are gaining or losing at each point score.

Multipliers gone wild

In a tour de force of mathematical derring-do, I’m going to take these three numbers and multiply them together.

The “difference” from the previous table tells us how much better or worse players are serving at a specific point score, compared to their overall performance. If two differences are similar, the one that matters more is the one with higher volatility, right? So we multiply by volatility. And all else equal, the more often a situation occurs, the greater its impact on the end result. So we multiply by the number of occurrences in the dataset.

The final tally is volatility * occurrences * difference, cleverly dubbed “V*O*D” in the table below. The product of three percentages is tiny, so I’ve multiplied those figures by 10,000 to make the results easier to read.

Here are the results:

Score  Volatility  Occurrences  Difference  V*O*D  
40-AD       76.3%          22%       -1.8%  -29.9  
0-0         22.5%         100%       -1.3%  -29.2  
15-30       44.9%          34%       -0.4%   -5.8  
15-0        16.5%          50%       -0.6%   -4.9  
40-30       23.8%          26%       -0.6%   -3.6  
40-40       42.5%          43%       -0.1%   -2.6  
0-15        33.2%          50%       -0.1%   -2.3  
30-0         9.3%          27%       -0.3%   -0.9  
                                                   
Score  Volatility  Occurrences  Difference  V*O*D  
40-15        8.5%          24%        0.0%    0.1  
30-15       20.7%          34%        0.1%    0.6  
AD-40       23.8%          22%        0.2%    1.1  
40-0         3.0%          16%        3.4%    1.7  
30-30       42.5%          32%        0.4%    5.9  
0-40        31.4%          16%        1.4%    7.1  
0-30        40.0%          27%        0.8%    8.2  
15-15       29.4%          46%        0.8%   11.0  
30-40       76.3%          25%        1.4%   26.3  
15-40       49.0%          24%        2.4%   28.2

With all factors taken into account, we see that servers are giving up about as much on the first point of the game as they are when faced with nerves at 40-AD. Two point scores also stick out at the other end of the spectrum, where 30-40 puzzlingly continues to be a time when servers find their best stuff.

Exploiting the mundane

The exact V*O*D numbers are far (far!) from natural laws, but when I ran the same algorithm on data from other grand slams, the contours were nearly the same. In the 2017 and 2018 US Opens, for instance, 40-AD and 0-0 were again the standout “underperforming” points, and 0-0 was the one that topped the list.

* I took a rudimentary look at this topic very early in the blog’s history, using data from 2011. 0-0 didn’t stick out to the same degree, but I didn’t control for the deuce/ad difference, as I have today. When accounting for deuce-court strength, 0-0 performance looks relatively worse.

All of which is to say: I can’t explain why this is a thing, but it sure looks like it’s a thing. And if it’s a thing, it looks like an opportunity for savvy players and coaches.

I’m perfectly happy to accept that servers struggle to maintain their focus (and perhaps their ability to surprise) at 40-AD. More importantly, I’m sure that players and coaches are very aware of the necessary mental gymnastics so deep in a game.

On the other hand, there’s no good reason that servers should underperform at the start of every game. In fact, I’d be more ready to accept the idea that servers would have the edge. The opponent hasn’t seen a serve for a few minutes (or more), and the server’s arm is (relatively) fresh. While it’s not a recipe for domination, it sounds like a recipe for a tiny edge that the server can build on.

That’s why I believe there’s something to be exploited here. Perhaps players–or at least some of them–are taking a bit off their first-point first serves, using the opening salvo as a mini-warmup. Maybe they are more willing to hit their second-best serve, or aim to the returner’s stronger side, as a tactical move to set up more effective serves later in the game. As I’ve said, I don’t know why the numbers are turning up this underperformance, but it’s clear there’s a gap to be closed.

There’s no magic in the first point, but there’s an awful lot of value. Players who serve up their best stuff at the beginning of the game are getting an edge that their peers ought to be developing, too.

So You’d Like To Do Some Tennis Research

Great! Here’s some data.

Maybe you’ve got a class project that will allow to you pick your own dataset. Or perhaps you just think that tennis analytics are cool, and you’d like to jump in. One of the more common questions I get is from people in this situation who are looking for a little guidance in choosing a subject. Here are a few tips.

1. Scratch your own itch

I try not to pick topics for others, because I generally find that people do better work (and are more likely to stick with it) when they are “scratching their own itch,” working on what they find particularly interesting. If nothing comes to mind, keep reading.

2. Get skeptical

When you’re watching tennis or reading about it, get in the habit of questioning everything. Does that player really hit more wide serves on break points? Does that guy really play better when he’s leading? If you listen with this type of mindset, you can come away from watching a single match with half a dozen new ideas.

This tip presupposes what might be step 0 — watch and read about tennis! I assume that if you’ve found my blog and want to do analytics, you’re already a pretty big fan. Keep it up–any analyst can benefit from attentively watching more tennis. Reading analytical work is also key, both to get ideas, and to learn what effective studies look like.

3. Think analogically

Many of us who do tennis analytics also work in other sports. Others are academics such as economists and statisticians whose “real jobs” have them working in fields far from athletics. Non-tennis subjects aren’t irrelevant–quite the contrary! If you do an interesting hockey study, or read about an interesting experimental design in development economics, think about how else you could apply a similar approach. Sometimes it’s a dead end with no direct application to tennis, but the exercise itself has value–practicing this kind of thinking eventually pays off.

This tip can be particularly useful for those of you doing a class project. If your professor provides examples of the type of work they’d like to see, consider if there’s a close cousin in tennis analytics. That first thought might not be where you end up, but it’s a good way both to get ideas and to ensure that you’re doing roughly the sort of work that’s asked of you.

4. Chart a match (or ten)

The Match Charting Project is the largest public dataset of shot-by-shot tennis data. It can be overwhelming at first, so if you are considering doing research with the dataset, I strongly recommend charting a match or two as a way to get familiar with it.

Charting a match is also a great way to generate more questions. It forces you to watch closely, so you’ll notice tactics that you might not have otherwise seen. As you chart, you might find yourself dreaming up hypotheses–say, that a player’s service return is particularly effective when she steps inside the baseline. The rest of the match will offer more data to confirm or contradict, and it might help you develop more ideas about where to go from there.

5. Collect your own data

There’s more than enough tennis data out there to keep you busy for a very long time. But don’t be afraid to strike out in a new direction. Perhaps you’d like to study whether certain players are more effective under the lights, which would require tracking the start time of matches. Maybe you’d like to see if certain coaches are particularly good at extracting better performances from their charges, which means you’d need to build a database of coaches, look up when they worked with each of their players, and how the players fared during that time.

Many analysts think that their job is just that–analysis. But in some areas, there more to be gained from better data than from better analysis. Plus, building a new dataset doesn’t have to be a monumental task. The coaches example I gave might include only a few dozen coaches, who worked with a handful of players each.

6. Start small

Following some of my suggestions above can lead you into a huge, ambitious project. the most common result of taking on a huge project is an unfinished project, as I can tell you from experience. Before going big, try to find a “proof of concept” both to get your feet wet, and to see whether you’re on a useful track.

In the coaches example I just gave, you might look at what happened to the WTA rankings of Wim Fissette’s players when they worked with him. I don’t know if there’s a “Fissette effect,” and now that I mention it, I’m curious! That’s a mini-project you could do in an afternoon, and it gets you started on the path of a more thorough study.

Ok, ok, here’s a list

Still stuck? A few years ago, Carl and I put together a list of potential research topics. I’ve since taken it down, but Peter forked it, so it still exists on GitHub.

Some of the topics have already been done, and several others are beyond the scope of what’s possible with publicly-available data. That still leaves you with dozens of ideas.

Finally, once you’ve completed a study–big or small–be sure to post it on twitter and share with other tennis analysts. Your work might be the key that gives the next graduate student or hobby analyst the spark to start a project of their own.

Are Tournament Draws Giving Us Suspiciously Many Venus-Serena Clashes?

This week in Lexington, top seed Serena Williams faces her sister, Venus Williams, in the second round. They are both among the all-time greats, and they have played each other nine times in grand slam finals, so it’s always jarring to see them turn up in the same section of a draw and play on a Thursday.

Lately, their encounters seem to always happen long before the business end of a tournament. Their three matches between the 2017 Australian Open final and this week in Lexington all happened in the round of 32, including a planned 2019 Rome meeting from which Serena withdrew. Venus is usually unseeded, no longer the world-beater she once was, so it is at least possible that the Williams sisters would be bracket neighbors in any given week.

But should it happen quite so often? It is an understatement to say that Serena and Venus were not universally embraced upon arrival in the tennis world. If you’re conspiracy minded, every tournament draw is an opportunity to commit dastardly deeds. Perhaps early in the Williams era, it was the work of racist or otherwise misguided tournament officials who wanted to avoid all-Williams finals. Or nowadays, event honchos recognize that Venus is unlikely to reach the final, so they tinker with the bracket to make a headline-grabbing Williams-versus-Williams clash more likely.

I’m sure that most draws are conducted on the up-and-up, but the process is sufficiently opaque that it’s easy to get suspicious. It’s also easy to make mistaken generalizations from insufficient data. Let’s see what the numbers can tell us.

150 tournaments!

Lexington is the 150th tour event with both Serena and Venus in the field.*

* I think. My WTA data isn’t perfect for the early years of their careers, and there was an uncomfortable amount of manual tabulation involved in this post. Their TennisAbstract player pages are missing the 1999 Grand Slam Cup, but I’ve included it in all the numbers here. For the purposes of doing analytics, it doesn’t matter much if the total is 148 or 151, but if you’re printing a banner or making a cake, you should double-check.

Thursday’s match in Lexington will be their 31st, plus one withdrawal apiece. In 13 of the 150 events, the Williams sisters were either the top two seeds or the 3rd and 4th seeds, meaning that draw shenanigans were out of the question–they could not face each other until the final. 4 of those 13 times, that’s exactly what they did.

What are the odds?*

* Of me being able to use this sub-heading in any given blog post?

I went through the remaining 137 tournaments and identified the round in which they either did meet or could have met. For the purposes of analyzing draws, there isn’t really a difference. For instance, Serena and Venus have landed in the same half 73 out of a possible 137 times, a bit more than the 68 or 69 times that we would expect.

Because of their seeds, they had the chance of ending up in the same quarter 116 times, and that’s how it worked out 28 times, just under the 29 times that an exact one-in-four rate would’ve given them. The smaller the draw section, the fewer tournaments that Serena’s and Venus’s seeds made it possible for them to meet.

I counted the number of tournaments with a possible meeting on or before a certain round, and then the number of events in which the draw delivered that meeting, regardless of whether both Williamses got that far. Here are the results, along with the probability of that many or more actual meetings:

Section  Possible  Actual  Chance  
Half          137      73     25%  
Quarter       116      28     62%  
Eighth         85      17      3%  
16th           64       5     37%  
32nd           42       1     74%

There’s a one-in-four chance that Serena and Venus would’ve landed in the same half as many times as they have throughout their entire careers. That’s a bit of bad luck, but it’s hardly a smoking gun. The same is true for the same quarters, as well as very early meetings that would pit them against each other in the round of 32 or 64.

That leaves one eyebrow-raising number to discuss. On 85 occasions, at least one of the two women was seeded outside the top eight, making possible a meeting in the round of 16 or earlier. Given random draws, we’d expect 10 or 11 brackets in which they could face each other so early. Instead, we got 17.

A 3% chance of so many early encounters isn’t quite as bad as it sounds. I’ve tried to walk you through this process in the way I approached it. While I wondered if Serena and Venus have met more often than random draws would normally deliver, I didn’t have a particular round in mind. As you’ve seen, I generated a bunch of numbers, and one of the five looked suspicious. You might be able to construct a story that explains why the round of 16 is different from the others (such as my theory that tournament directors want mid-week headlines), but because we generated so many numbers, we were that much more likely to end up with an extreme percentage simply by chance.

The smoking (nerf) gun

Thus, we’re able to raise the possibilities that some draws weren’t random, but we can hardly prove it. One problem–one that we could’ve foreseen from the get-go–is that some draws are definitely not tampered with. Probably most draws. And even if they were, most tournaments wouldn’t have any reason to mess with Serena’s or Venus’s placement in the bracket. Or if they did, they might prefer an all-Williams final, and thus alter the bracket in the opposite direction of what we’re hunting for.

If you like conspiracy hunting, I’ve got a tiny sample for you. Since the beginning of 2018, Venus and Serena have played in the same tournament 15 times, and their seedings (or lack thereof) made it possible for them to be drawn in the same eighth 14 of those times. Of the 14, they were placed in position for a round-of-16 or earlier meeting 5 times. There’s only a 2% chance of that … if you set aside the fact that I’m checking all sorts of subsets of matches looking for (probably spurious) patterns. If nothing else, the 5-of-14 figure explains why it seems like Serena and Venus keep landing in the same draw sections lately. They do!

Broadly speaking, then, this is all much ado about nothing. (I don’t even know if these conspiracy theorists exist, so maybe I just invented a conspiracy and spent my evening debunking it. Hooray?) It’s possible that a few tournament directors are producing non-random draws … but it would take a very different kind of investigative work to prove it. Worst case scenario, we get a few more Serena-Venus matches. It may not be fair to the older sister, but it’s a pretty good deal for tennis fans.

Visualizing Trends in Net Play Across Five Decades of Grass Court Tennis

Earlier this week, I wrote about one aspect of the long-term decline in net play: the widespread belief that approaching the net is more difficult now because fewer players have a weaker side. I presented evidence indicating that most players still have a weaker side, which suggests that all groundstrokes–on both strong and weak sides–have gotten stronger, making net play a riskier proposition.

If that is true, it is reasonable to assume that passing shot winners are more frequent (relative to the number of net approaches), and perhaps that volleys are more aggressive, resulting in more first-volley winners and first-volley errors. More powerful and precise strokes should, on balance, make net points shorter than they used to be.

We can begin to test these theories using the extensive shot-by-shot records assembled by the Match Charting Project (MCP). MCP data includes every men’s Wimbledon final and semi-final back to 1990, as well as many elite-level grass court matches from the 1970s and 80s. For the purposes of today’s study, I will use only Wimbledon semi-finals and finals, plus a handful of other grass court matches from 1970-89 to complement the sparser Wimbledon data. This way, we know we’re comparing the elites of various generations to one another.

Contemporary net approaches

Let’s start by looking at what happens in a 2010s Wimbledon’s men’s final or semi-final when a player approaches the net. I’m excluding serve-and-volley points, and will do so throughout. I’m also excluding approach shot winners, which are often little more than gestures in the direction of the net following a big shot. (Even when they’re not, it can be difficult for charters to distinguish between approach and non-approach winners.) Thus, we’re looking at about 1,250 net approaches in which the other player got his racket on the ball.

The ball came back almost 73% of the time, and on slightly more than half the points, the approaching player put his first volley (or smash, or whatever shot he needed to hit) in play. 19% of the points saw a second passing shot attempt put in play, and nearly 12% had a second net shot keep the point going. About 1 in 30 approach-shot points continued even longer, forcing the the netman to contend with a third pass attempt.

The following visualization is a Sankey diagram showing how these net points developed. “App” stands for approach, “Unret” for “unreturned,” “Pass1” for “first passing shot,” “V1” for “first volley,” and so on. Mouse over any region of the diagram for a brief summary of what it represents.

There’s a lot of information in the graphic, and it may not be entirely intuitive, especially hindered by my clunky design. Each region is sized based on what fraction of points developed in a certain way. As the regions move toward the right side of the diagram, they as classified by whether the approaching player won the point. As we can see, in the 2010s sample, these approach shots resulted in points won about 69% of the time.

The golden era

To compare eras, we need more than just one decade’s worth of data. I separated the approach shots by decade (grouping together the 70s and 80s), and the most distinctive era turned out to be the 1990s, when Pete Sampras ruled the roost and many of his challengers were equally aggressive.

Far more points were opened with a serve-and-volley: almost 81% in the 1990s compared to 7% in this decade. Even with the server claiming the net so early and so often, there were still many more non-serve-and-volley net approaches two decades ago. Then, there were about 85 “other” net approaches per match; this decade, there have been about 27. Thus, it is reasonable to assume that the typical net approach started from a less favorable position. These days, players only approach when the point has developed in a particularly inviting way.

Here is another diagram, this one showing what happened following 1990s net approaches:

It’s striking to see that, back when net play was much more common, with a master such as Sampras dominating our sample, net approaches were less successful than they are today, resulting in a 67% win rate instead of 69%. However, it’s tough to know how today’s players–even a confident aggressor like Roger Federer or a volleying wizard like Rafael Nadal–would fare if they came forward four times as much. Assuming they pick their spots wisely, their success rate would be lower than 69%. The only question is how much lower.

Contrary to my inital hypothesis, passing shots seemed to be higher-risk and higher-reward in the 1990s than in the 2010s. Two decades ago, only 65.5% of initial passing shot attempts were put in play (compared to 72.6% today), though nearly as many of those attempts resulted in winners (21.1% to 21.4%). It was the volleyers who were either more conservative or less powerful in the 1990s. Then, barely half of first volleys ended the point in the netman’s favor; now, the number is closer to 60%. Again, this could be because today’s players pick their spots more carefully, allowing them to hit easier first volleys.

The early days

We’ve seen how net approaches developed in the 1990s and the 2010s. It would be reasonable to assume that the 1980s (with several late ’70s matches thrown in) were like the 1990s, but more so. Instead, the results are more of a mixed bag, with some characteristics that look like the ’90s, and others that are closer to today’s numbers.

Here is the diagram:

In the 1980s, nearly as many passing shot attempts were put in play as they are today, in contrast to the lower rate during the 1990s. First volleys are a similar story. When passing shot attempts came back, approaching players put a volley (or other net shot) back in the court about 70% of the time–similar numbers in the 1980s and 2010s, but a couple percentage points higher than in the 1990s.

What is different is what happened next. In the 1980s, if the approaching player put his first volley back in play, it came back again 53% of the time. That rate is one of the few clear trends over time: It fell to 46% in the 1990s, 45% in the 2000s, and 37% in the 2010s. As a result, the ’80s saw far more second volleys and points that extended even further, compared to more recent eras. The lack of first-volley putaways meant that net approaches only converted into points won about 65% of the time.

A cautious narrative

There is no simple explanation that accounts for all of these numbers, because we are not seeing the direct result of a single factor, like the shift from wooden rackets or to more topspin-friendly string. Technological changes certainly have an impact, but as soon as the balance between approacher and opponent shifts, players adjust their strategy accordingly.

For instance, the rate of points won on net approaches appears to have steadily increased, from 65% in the 1980s to 67% in the 1990s to 69% today. The first rise could be attributed to racket technology, which gave aggressors more power and control. But the second rise came over a time period in which string technology offered more help to defenders. The higher rate of approach points won isn’t because players are better at it, it’s because they picked their spots more carefully.

What we might focus on instead, then, is how much these diagrams look alike, even though they represent vastly different eras. While there isn’t exactly a net-approach-strategy equilibrium that has held through the decades, player decision-making has kept these rates from varying too wildly. If passing shot winners start going up, we’ll probably see even fewer approaches–with the remaining approaches in still more favorable moments–or a continued increase in the percentage of approaches to the backhand side. That’s another clear trend over the last few decades, but it’s a topic for another day.

Rather than succumbing to nostalgia and bemoaning the decline of net play, it’s better to celebrate the adaptability of tennis players at the highest level. While the game a whole has become more defensive, backcourt denizens from Bjorn Borg (94 approaches per charted grass-court match) to Novak Djokovic (21 approaches per match) have reminded us that adjustments work in both directions. With parameters such as technology, surface, and opponent skills constantly changing, we can’t expect winning strategy to remain the same.

Thanks to SankeyMATIC for making it easy to create the diagrams.

Nick Kyrgios Really Is Different Under Pressure

Italian translation at settesei.it

Earlier this week, we looked at whether Nick Kyrgios is unusually inconsistent. That is, is he more likely to upset higher-ranked players and lose to lower-ranked players than his peers? The numbers say he isn’t.

But that isn’t all we mean when we talk about Kyrgios’s unreliability. He often undergoes dramatic shifts within matches. At times, he is visibly distracted; during his Delray Beach match against Radu Albot, he even shouted that he wanted to get off the court. Other times, he comes up with breathtaking serving and shotmaking at the most crucial moments. He seems motivated by both packed grandstands and on-court pressure. Unfortunately, both of those are missing from a lot of professional tennis.

We already have some evidence for the better-under-pressure hypothesis. In his five matches in Acapulco last week, he won a mere 50.4% of points, one of the lowest totals ever for a title-winner. In three of the five matches, he won return points at a lower rate this opponent, resulting in Dominance Ratios (DRs) below 1.0. Winning a match with a sub-1.0 DR (or fewer than 50% of total points won) isn’t unheard of, but it’s not a reliable way to rise to the top of the sport. Such contests are called “lottery matches” for a reason–there’s a lot of luck involved in winning with such fine margins, and fortune tends to even out.

Yet Kyrgios’s “luck” keeps nudging his results in the same direction. He has played 15 career tour-level matches in which his DR is between 0.9 and 0.99–close matches in which he was slightly outplayed, at least in the points column. With stats like that, players tend to win about one-third of the time. Kyrgios, however, has won eleven of those 15 matches. His good fortune doesn’t cancel out when he narrowly edges out an opponent: In 13 matches with DRs between 1.0 and 1.1, he has lost only two. The Australian is doing something right.

Big points are big

You probably already know what’s going on here, even if you haven’t listened to commentators speculate during Nick’s matches. The key to such narrow victories is converting the “big” points–break points, deuces, tiebreaks, and so on. It doesn’t matter if you throw away a point or two when serving at 40-love. Other situations have considerably more leverage, and that’s when Kyrgios brings his best tennis.

I tallied up Kyrgios’s return points won over the course of his career, based on the point score of each one. (I don’t have the point-by-point sequence of every one of his tour-level matches, but most of them are included, more than enough to constitute a reliable sample.) Here are the five games scores when he wins the most return points, starting with the most effective:

0-40, 40-AD, 15-30, 30-40, 40-40

And the five scores, again in order, starting with least effective:

30-0, 40-0, 40-15, 0-15, 0-0

In other words, when he has a chance to break, he’s great. In my sample of matches, he won 31.5% of return points; when the opposing server is facing him at 0-40, he wins the point 45.0% of the time. At 40-AD, it’s 41.9%. When his opponent serves with a 30-0 advantage, Kyrgios wins a mere 27.3% of return points.

Everybody does it (a little)

Astute readers will realize that I haven’t accounted for a key variable. In a data set of dozens of matches, scores that favor the returner will occur more often against weaker servers. Kyrgios didn’t get many 0-40 or even 40-AD chances against John Isner last week, but he can expect to get more against the likes of Albot. So to some extent, we should expect players to win more return points at these moments. In the last 52 weeks, ATPers have won 37.3% of return points, but 40.1% of break points.

Everybody does it, but Nick does it more. The following table shows the ratio of return points won at each game score to average return points won. The middle column shows Kyrgios’s ratios and the right-most column shows the 2018 ATP tour average:

Situation       NK   ATP  
0-40          1.43  1.14  
40-AD         1.33  1.09  
15-30         1.27  1.05  
30-40         1.26  1.06  
40-40         1.16  1.02  
15-40         1.13  1.06  
15-15         1.11  0.99  
15-0          1.11  0.98  
30-15         1.09  1.00

Situation       NK   ATP  
0-30          1.07  1.06 
AD-40         1.06  1.02  
40-30         1.05  1.00  
30-30         1.03  1.01  
0-0           1.02  0.99  
0-15          1.01  1.05  
40-15         0.95  0.92  
40-0          0.91  0.87  
30-0          0.87  0.91

Most players take advantage in 0-40 situations, and to a lesser extent at break points, but Kyrgios is on another planet. The average player wins roughly 10% more return points in break situations; Kyrgios triples the ratio.

Leverage

We’ve taken a big step toward explaining Kyrgios’s pattern-breaking results and his in-match inconsistency. But even game scores don’t tell the whole story. A deuce point at 5-0 usually matters a great deal more than a break point when the returner is already up a set and a break.

To account for those differences, we’ll turn to the leverage metric. (You’ll also see it referred to as “volatility” or “importance.”) Here’s the idea: Given what we know about two players, we can calculate the probability that one of them will win the match, based on the current situation. If the server wins, that probability shifts in his favor. If the returner wins, it shifts in the opposite direction. Leverage is the sum of those two shifts: the amount of win probability that is at stake at any given point.

For today’s purposes, there are no specific numbers; you need only to understand the concept. The higher the leverage, the more the point matters. Players might disagree with some of the details that a purely math-based approach spits out, but for the most part, the equations capture our intuition about which points matter, and how much.

I calculated the leverage for every point of the 2018 ATP season and split the points into ten categories, from least important (1) to most important (10). The following graph shows the tour average rate of return points won (RPW) for each of those ten categories:

If we ignore the leftmost and rightmost data points, there’s something of a trend here. From the second-to-least-important category to the second-to-most-important, players increase their return points won from about 36.0% to 37.5%. Some of that shift can be explained by a phenomenon I’ve already mentioned: returners find themselves in crucial situations (such as break points) more often against weaker servers.

Here’s the same graph, now with a second line showing Kyrgios’s RPW in the ten categories, from least important to most important. I’ve kept the ATP average trendline for comparison:

Remember that 36.0% to 37.5% increase I mentioned a minute ago? For Kyrgios, the same shift is 27.0% to 35.2%–eight percentage points instead of less than two. It appears that the Australian is extremely sensitive to what’s at stake throughout matches, and when the rewards are high enough, he turns into a credible returner.

Some of you are probably thinking, “of course, I knew that all along.” First of all, I hate it when people say that, because what they really mean is, “I suspected that all along,” and they didn’t really know. Some of the other things such people “know” are actually wrong.

Second, I need to underline just how unusual this is. I’ve been playing around with point-by-point data for a few years now, looking for in-match patterns, for specific players and for the sport overall. Such patterns exist: points and games aren’t entirely independent of each other. But usually they are minor–a percentage point or two, not the kind of thing you could spot even in a fortnight’s worth of matches. Kyrgios breaks the mold. When it comes to the mercurial Australian, the assumptions that are adequate to account for most of professional tennis simply fail.

Ivo Karlovic’s Survival and the Key to Aging in Men’s Tennis

Italian translation at settesei.it

Let’s just get this out of the way first: Ivo Karlovic is amazing. The Croatian didn’t play his first tour-level match until he was 22 years old, and he didn’t crack the top 100 for two years after that. Yet he eventually reached No. 14 in the world, won over 350 career matches, and claimed nine tour-level titles. Now, a few weeks shy of his 40th birthday, he’s coming off an ATP final in Pune, where he came within two points of ousting top-ten stalwart Kevin Anderson and ensured that he’ll remain in the top 100 through his milestone birthday next month.

The fact that Karlovic is one of the tallest men ever to play the game and that he holds a wide array of ace records is beside the point. (Though it’s certainly worthy of discussion, and I hope to dive into aging patterns and playing styles in a future post.) Yes, his first-strike brand of tennis, avoiding the bruising rallies that have worn down the likes of David Ferrer, may make it easier to compete at an advanced age. On the other hand, he remains of the few men on tour to regularly serve-and-volley, a tactic that scores of younger, quicker men can’t execute effectively. He is, quite simply, one of a kind.

Despite his uniqueness, Karlovic represents an important aspect of men’s tennis in the 2010s. The ATP has gotten older since he broke in almost two decades ago, and ten men aged at least 33 are ranked higher than the Croatian. One of them, 37-year-old Roger Federer, remains one of the best players in the game. The average age of elite men’s tennis may be creeping back down, but it is still the golden age of 30-somethings.

Men like Karlovic and Federer have seemed to defy the usual logic of aging. Most sports have a reliable “peak age” at which players can be expected to to perform their best. Up to that point, competitors are developing both physically and mentally; after the peak age, physical deterioration sets in and performance declines. There’s always plenty of variation around the average, but the overall trajectory–break in, rise, peak, fall, retire–is predictable enough.

In part, Karlovic has followed that path, just with a late start and a surprise second peak in his 30s. To compare year-to-year performances, I calculated each player’s dominance ratio (DR), a useful measure of overall performance calculated as the ratio of return points won to opponents’ return points won, and adjusted it for quality of competition. (The adjustment algorithm gets complicated; I first outlined how it controls for each player’s mix of opponents here.) 1.0 is average, and the typical range runs from about 0.8 (soon to head back to challengers) to 1.2 (big four territory). The following graph shows Ivo’s DR at each age, along with a smoother three-year moving average:

Karlovic hit his primary peak around age 31, a bit late but not entirely atypical for the era. Even if we ignore the surprise spike at age 36, he remained an average player (roughly speaking, a card-carrying member of the top 50) until age 35. In 2017 and 2018, we finally witnessed a downward trend, but if Ivo’s feat in Pune is any indication, he might be turning things around once again.

Nearly every professional tennis player retires before they reach Karlovic’s current age, so we’ll never know what bonus peaks we missed. Of course, many of those retirement decisions are due to injury, so at least some of the Croatian’s late-career success must be credited to his ability to stay healthy enough to soldier on. Let’s look at an even more baffling aging pattern, one that belongs to a player who will almost definitely retire before seeing the kind of late-career decline that Karlovic experienced in 2017 and 2018. Here’s Federer:

By the measure of competition-adjusted DR, Federer’s best season came at age 34. Even if you don’t buy that, the overall trend is clear. He continues to play at or near his peak, past the age at which his peers become Davis Cup captains and have Tour Finals round-robin groups named after them.

Federer has been able to stay off the injured list for almost all of his 20 years on tour, and health–the simple fact of showing up for most tournaments–may be the most underrated skill in men’s tennis. The vast majority of players who don’t survive to post elite seasons in their mid- and late-30s aren’t slowly drifting down the ranking list, like a baseball player who plays every game in his 20s, then moves into more and more limited part time roles as he ages. Instead, they drop out, perhaps because of a single career-ending injury, the general accumulation of nagging problems, or lack of desire to wholeheartedly pursue the sport at the expense of everything else.

The following graph shows the two ways in which players fail to maintain their previous level from one year to the next: Playing worse tennis (measured by competition-adjusted DR), or leaving the tour. The latter is defined by contesting fewer than 20 tour-level matches, something that any reasonably healthy player with a ranking in the top 100 should be able to manage. At every age, players drop out at a surprising clip, and that rate begins to overtake the percentage of players who stay on tour but perform at a weaker level around the late 20s:

The “Leave tour” rates slightly overstate the number of disappearing players, since about one-quarter of them eventually return to the tour, like Andy Murray is trying to do in 2019. But even accounting for the number of comebacks, a hefty share of the players we expect to steadily decline are either forced off tour by injury or choose not to continue.

Selection bias

All of these disappearing players make it extremely difficult to construct an aging curve for men’s tennis. One common approach to measuring such a trajectory is to identify all the players who competed in consecutive seasons (say, their age-25 and age-26 campaigns), figure out how much better or worse they performed in the latter year, and average the differences. When we do that for ATP players born since 1970, the results are downright bizarre. The worst year-to-year change is from age 21 to age 22, when DR decreases by about 2.3%, even though we would expect youngsters to be developing their game for the better. The strongest year-to-year change is from age 30 to age 31, with an improvement of 4.0%, when we would expect a plateau or even a slight decline.

Because these ratios don’t include the players who drop out, most of the year-to-year ratios reflect an improvement:

Age       Year-to-year DR ratio  
19 to 20                  -1.7%  
20 to 21                  +0.9%  
21 to 22                  -2.2%  
22 to 23                  -0.3%  
23 to 24                  +1.5%  
24 to 25                  +1.1%  
25 to 26                  +0.7%  
26 to 27                  +1.5%  
27 to 28                  +1.2%  
28 to 29                  +3.5%  
29 to 30                  -0.8%  
30 to 31                  +4.0%  
31 to 32                  +2.6%  
32 to 33                  +0.7%  
33 to 34                  -0.5%  
34 to 35                  +3.0%  
35 to 36                  -0.4%

If we assembled these ratios into an aging curve, we’d see a line staggering upwards, as if we could expect players to continue improving for as long as they cared to compete.

However, things start to make sense when we acknowledge the selection bias and reframe our findings accordingly. It isn’t true to say that the average player steadily improves forever. But it is more believable to say this: The average player who remains healthy enough to play a full season and has the desire to compete full-time can expect to improve well into his 30s. The older a player gets, the less likely that the second claim applies to him.

As they say, half of success is just showing up. By age 39, most pro tennis players have long since started showing up somewhere else. By dint of sheer perserverance, a bit of luck, and one of the most dominant serves the world has ever seen, Karlovic has shown that tennis’s aging curve is even more flexible that we thought.

Dominic Thiem In Pressure Service Games

Embed from Getty Images

Dominic Thiem has good reason to be frustrated.

Italian translation at settesei.it

On Tuesday night, Rafael Nadal and Dominic Thiem delivered the match of the 2018 US Open thus far. After nearly five hours of play, nothing separated them as they battled their way to 5-5 in a fifth-set tiebreak. Nadal finally crept ahead by the narrowest of margins, sealing a victory by the unlikely score of 0-6 6-4 7-5 6-7(4) 7-6(5).

Both players had plenty of chances, and while Rafa prepares for a semi-final against Juan Martin del Potro, Thiem will have plenty of time to mull over the opportunities he missed. In the second set, he failed to hold in both of his last two service games, including the final frame of the set, at 4-5. In the third set, he took the lead by breaking Nadal in the seventh game, but failed to follow up his advantage, losing serve when he attempted to serve it out at 5-4. Two games later, he proved unable to hold serve to stay in the set at 5-6, though he forced Rafa to four deuces before finally giving way.

These three missed chances are hardly the entire story of the match, but they stick out in memory. Overall, Thiem served quite well, allowing Nadal only one break per set. That’s 21 holds in 26 service games, an 81% hold rate, a significant achievement compared to the 66% that Nadal’s opponents have averaged against him on hard courts this year, or the paltry 52% that Rafa has allowed overall. The problem isn’t that the Austrian served badly–he didn’t–but that he weakened at the wrong times. Thiem broke Nadal more often than Rafa returned the favor–six to five–but because three of Thiem’s breaks came in the first, 6-0 set [editor’s note: !??!?!?!?] , Nadal’s six proved less costly than Thiem’s five.

Bad day, or just bad?

Is this something Thiem does, or is it just something that he did, perhaps nudged over the edge one of the greatest returners of all time? Too often, viewers–along with many of those paid to talk and write about tennis–see the latter and assume the former. Does Thiem make a habit of serving strong in lower-leverage games and then wilting when the pressure ratchets up?

If he does, it would make him an exception. I looked at “serving for the set” opportunities a few years ago and found that ATP players serve almost exactly as well when a hold would earn them the set than otherwise. The difference is a mere 0.7%, meaning that the “difficulty” of serving for the set translates into one additional break per 143 opportunities. The effect wasn’t any more noticeable when I narrowed the focus to situations in which the player led by only a single break, like Thiem’s dropped service game at 5-4 in the third set last night.

Let’s look again, and pay specific attention to Thiem. My dataset of sequential point-by-point data, spanning most ATP tour matches between late 2011 and a few weeks ago, now covers over 400,000 service games, including 30,000 serving-for-the-set chances, over two-thirds of them with a lead of a single break. Over 1% of them have Thiem serving, so at least our sample size benefits from the Austrian’s strenuous schedule, even if it doesn’t do him any favors on the court. In other words, we’ve got a ton of data here, so if there is an effect, we should be able to find it.

Thiem’s missed chances included chances to both finish a set and stay in a set, so I’ve expanded our view to a variety of pressure situations. For each situation, I’ve calculated the hold rate for players in that position relative to their typical hold rate in those matches. (A player with a lot of serve-to-stay-in opportunities is probably on the losing end, with a lower hold rate than average, but this method should control for that.) A ratio of 1.0 means that the hold rate in the pressure situation is exactly the same as normal. A ratio above 1.0 means the hold rate is higher than usual, and below 1.0 signifies a lower hold rate–the lag many of us expect to see when the stakes get higher. Here are the ratios for a variety of situations, including serving for the set (plus a category one-break leads), serving to stay in the set (also with one-break deficits identified), ties late in the set such as 4-4 and 5-5, and for comparison’s sake, low-pressure situations–“All Else”–which is a catch-all for everything not in the above categories.*

* Yes, it includes the famous seventh game, which I’ve previously shown isn’t particularly important, no matter what Bill Tilden said.

Situation          Examples  Hold% / Avg  
For-Set            5-4; 5-2        0.994  
- For-Set Close    5-3; 6-5        0.989    
To-Stay            4-5; 1-5        0.999  
- To-Stay Close    5-6; 3-5        0.969    
Tied Late          4-4; 5-5        0.953  
All Else           2-3; etc        1.003

The “serving-for-the-set” effect is almost exactly same as what I found three years ago: a drop of a bit more than half a percent. Last year, the impact of serving for the set with a single break lead was a bit greater than I initially found, but it’s still small. We find servers struggling the most when serving to stay in the set while trailing by a a single break–losing serve 3.1% more often than usual–and when serving at 4-4 and 5-5, when they drop serve almost 5% more frequently than expected. These are the most substantial effects I’ve seen, but keep in mind the magnitude–even a 5% difference means it only flips the outcome of one service game in twenty. It certainly matters, but it would be awfully hard to spot with the naked eye.

The one percent

How does Thiem compare? Here is the same set of ratios for him, with separate columns for his career numbers (subject to the limitations of my dataset, which includes few matches before 2012) and for single-season figures from 2016, 2017, and 2018:

Situation        Career   2016   2017   2018  
For-Set           0.996  1.049  1.011  0.966  
- For-Set Close   0.984  1.078  1.008  0.887  
To-Stay           1.030  1.160  1.027  0.940  
- To-Stay Close   0.984  1.148  0.957  0.964  
Tied Late         0.984  0.976  0.991  0.889  
All Else          1.004  0.994  1.009  1.030

Thiem’s career numbers reveal little, just a player who is a tiny bit worse in high-leverage situations, though perhaps a little less affected by the pressure than his peers. The concern is his numbers so far this year, which are way down across the board. Each one of the categories represents a relatively small sample–for example, I have only 42 games in which he was serving for the set with a single break advantage–but taken together, the set of sub-1.0 ratios don’t point in an encouraging direction. We could never have forecast before last night’s match that Thiem would serve so well in general but so much weaker in the clutch, but there were subtle hints lurking in his 2018 performance.

A puzzle

I want to show you the same set of data, but for another player. In one way, it’s the opposite of Thiem’s: many more breaks in pressure situations over the course of the player’s career, but the opposite trend in the last few years, pointing toward more service holds:

Situation        Career   2016   2017   2018  
For-Set           0.929  0.931  1.200  1.077  
- For-Set Close   0.910  0.895  1.333  1.000  
To-Stay           1.026  1.077  1.083  1.061  
- To-Stay Close   0.929  1.100  1.167  1.044  
Tied Late         0.905  1.050  1.000  1.048  
All Else          1.011  1.013  1.024  1.013

Any ideas? It’s a bit of a trick question–you’re looking at the tour serving against Rafa. From 2012-15, Nadal absolutely shut down opposing servers starting at about 4-4. (He wasn’t as good–relative to his average, anyway–late in sets on his own serve.) Very few players or seasons show effects of greater than 5% in either direction, but Rafa’s opponents saw their hold rate dip by more than twice that in some seasons. Yet the story has been different for the last year or two, with Rafa himself becoming the underperformer in his late-set return games.

Again, we shouldn’t read too much into a single year of this data: The sample size is an issue, especially for a top player’s return games, because not many guys find themselves serving for a set against him. But had we looked at Nadal’s return record in pressure situations alongside Thiem’s recent serve performance, it would have made for a more complicated picture, one less likely to predict some of the crucial moments in last night’s match. In any given contest, there are simply too few key games for us to forecast their outcome with any success, especially when a let cord, an untimely distraction, or a missed line call could reverse the result. But that doesn’t mean we shouldn’t try to understand them. Unlucky, unclutch, or whatever else, Thiem could have flipped the outcome of the entire match by holding just one of those three games. The stakes could hardly be higher.

Handling Injuries and Absences With Tennis Elo

Italian translation at settesei.it

For the last year or so, every mention of my ATP and WTA Elo ratings has required some sort of caveat. Ratings don’t change while players are absent from the tour, so Serena Williams, Novak Djokovic, Andy Murray, Maria Sharapova, and Victoria Azarenka were all stuck at the top of their tour’s Elo rankings. When their layoffs started, they were among the best, and even a smattering of poor results (or a near season’s worth, in the case of Sharapova) isn’t enough to knock them too far down the list.

This is contrary to common sense, and it’s very different from how the official ATP and WTA rankings treat these players. Common sense says that returning players probably aren’t as good as they were before a long break. The official rankings are harsher, removing players entirely after a full year away from the tour. Serena probably isn’t the best player on tour right now (as Elo insisted during her time off), but she’s also much more of a threat than her WTA ranking of No. 454 implies. We must be able to do better.

Before we fix the Elo algorithm, let’s take a moment to consider what “better” means. Fans tend to get worked up about rankings and seedings, as if a number confers value on the player. The official rankings are, by design, backward-looking: They measure players based on their performance over the last 52 weeks, weighted by how the tour prioritizes events. (They are used in a forward-looking way, for tournament seedings, but the system is not designed to be predictive of future results.) In this way, the official rankings say, “this is how good she has played for the last year.” Whatever her ability or potential, Serena (along with Vika, Murray, and Djokovic) hasn’t posted many positive results this year, and her ranking reflects that.

Elo, on the other hand, is designed to be predictive. Out of necessity, it can only use past results, but it uses those results in a way to best estimate how well a player is competing right now–our best proxy for how someone will play tomorrow, or next week. Elo ratings–even the naive ones that said Serena and Novak are your current No. 1s–are considerably better at predicting match outcomes than are the official rankings. For my purposes, that’s the definition of “better”–ratings that offer more accurate forecasts and, by extension, the best approximation of each player’s level right now.

The time-off penalty

When players leave the tour for very long, they return–at least on average, and at least temporarily–at a lower level. I identified every layoff of eight weeks or longer in ATP history, taken by a player with an Elo rating of 1900 or above*. In their first matches back on tour, their pre-break Elo overestimated their chances of winning by about 25%. It varies a bit by the amount of time off: eight- to ten-week breaks resulted in an overestimation around 17%, while 30- to 52-week breaks meant Elo overestimated a player’s chances by nearly 50% upon return. There are exceptions to every rule, like Roger Federer at the 2017 Australian Open, and Rafael Nadal, who won 14 matches in a row after his two-month break this season, but in general, players are worse when they come back.

* I used the cutoff of 1900 because, below that level, some players are alternating between the ATP and Challenger tours. My Elo algorithm doesn’t include challenger results, so for lower-rated players, it’s not clear which timespans are breaks, and which are series of challenger events. Also, the eight-week threshold doesn’t count the offseason, so an eight-week layoff might really mean ~16 weeks between events, with the break including the offseason.

Translated into Elo terms, an eight-week break results in a drop of 100 Elo points, and a not-quite-one-year break, like Andy Murray’s current injury layoff, means a drop of 150 points. Making that adjustment results in an immediate improvement in Elo’s predictiveness for the first match after a layoff, and a small improvement in predictiveness for the first 20 matches after a break.

Incorporating uncertainty

Elo is designed to always provide a “best estimate”–when a player is new on tour, we give him a provisional rating of 1500, and then adjust the rating after each match, depending on the result, the quality of the opponent, and how many matches our player has contested. That provisional 1500 is a completely ignorant guess, so the first adjustment is a big one. Over time, the size of a player’s Elo adjustments goes down, because we learn more about him. If a player loses his first-ever match to Joao Sousa, the only information we have is that he’s probably not as good as Sousa, so we subtract a lot of points. If Alexander Zverev loses to Sousa after more than 150 career matches, including dozens of wins over superior players, we’ll still dock Zverev a few points, but not as many, because we know so much more about him.

But after a layoff, we are a bit less certain that what we knew about a player is still relevant. Djokovic a great example right now. If he lost six out of nine matches (as he did between the Australian Open fourth round and Madrid) without missing any time beforehand, we’d know it was a slump, but most of us would expect him to snap out of it. Elo would reduce his rating, but he’d remain near the top. Since he missed the second half of last season, however, we’re more skeptical–perhaps he’ll never return to his former level. Other cases are even more clear-cut, as when a player returns from injury without being fully healed.

Thus, after a layoff, it makes sense to alter how much we adjust a player’s Elo ratings. This isn’t a new idea–it’s the core concept behind Glicko, another chess rating system that expands on Elo. Over the years, I’ve tinkered with Glicko quite a bit, looking for improvements that apply to tennis, without much success. Changing the multiplier that determines rating adjustments (known as the k factor) doesn’t improve the predictiveness of tennis Elo on its own, but combined with the post-layoff penalties I described above, it helps a bit.

The nitty-gritty: After a layoff, I increase the multiplier by a factor of 1.5, and then gradually reduce it back to 1x over the next 20 matches. The flexible multiplier slightly improves the accuracy of Elo ratings for those 20 matches, though the difference is minor compared to the effect of the initial penalty.

No more caveats*

* I thought it would be funny to put an asterisk after “no more caveats.”

Post-layoff penalties and flexible multipliers end up bringing down the current Elo ratings of the players who are in the middle of long breaks or have recently come back from them, giving us ranking tables that come closer to what we expect–and should do a better job of predicting the outcome of upcoming matches. These changes to the algorithm also have minor effects on the ratings of other players, because everyone’s rating depends on the rating of all of his or her opponents. So Taro Daniel’s Elo bounce from defeating Djokovic in Indian Wells doesn’t look quite as good as it did before I implemented the penalty.

On the ATP side, the new algorithm knocks Djokovic down to 3rd in overall Elo, Murray to 6th, Jo-Wilfried Tsonga to 21st, and Stan Wawrinka to 24th. That’s still quite high for Novak considering what we’ve seen this year, but remember that the Elo algorithm only knows about his on-court performances: A six-month break followed by a half-dozen disappointing losses. The overall effect is about a 200-point drop from his pre-layoff level; the “problem” is that his Elo a year ago reflected how jaw-droppingly good he had recently been.

The WTA results match my intuition even better than I hoped. Serena falls to 7th, Sharapova to 18th, and Azarenka to 23rd. Because of the flexible multiplier, a few early wins for Williams will send her quickly back up the rankings. Like Djokovic, she rates so high in part because of her stratospheric Elo rating before her time off. For her part, Sharapova still rates higher by Elo than she does in the official rankings. Despite the penalty for her one-year drug suspension, the algorithm still treats her prior success as relevant, even if that relevance fades a bit more every week.

Elo is always an approximation, and given the wide range of causes that will sideline a player, not to mention the spectrum of strategies for returning to the tour, any rating/forecasting system is going to have a harder time with players in that situation. That said, these improvements give us Elo ratings that do a better job of representing the current level of players who have missed time, and they will allow us to make superior predictions about matches and tournaments involving those players.

Under the hood

If you’re interested in some technical details, keep reading.

Before making these adjustments, the Brier score for Elo-based predictions of all ATP matches since 1972 was about 0.20. For all matches that involved at least one player with an Elo of 1900 or better, it was 0.17. (Not only are 1900+ players better, their ratings tend to be based on more data, which at least partly explains why the predictions are better. The lower the Brier score, the better.)

For the population of about 500 “first matches” after layoffs for qualifying players, the Brier score before these changes was 0.192. After implementing the penalty, it improved to 0.173.

For the 2nd through 20th post-comeback matches, the Brier score for the original algorithm was 0.195. After adding the penalty, it was 0.191, and after making the multiplier flexible, it fell a bit more to 0.190. (Additional increases to the post-layoff multiplier had negative results, pushing the Brier score back to about 0.195 when the 2nd-match multiplier was 2x.) I realize that’s a tiny change, and it very possibly won’t hold up in the future. But in looking at various notable players over the course of their comebacks, that’s the option that generated results that looked the most intuitively accurate. Since my intuition matched the best Brier score (however miniscule the difference), it seems like the best option.

Finally, a note on players with multiple layoffs. If someone misses six months, plays a few matches, then misses another two months, it doesn’t seem right to apply the penalty twice. There aren’t a lot of instances to use for testing, but the limited sample confirms this. My solution: If the second layoff is within two years of the previous comeback, combine the length of the two layoffs (here: eight months), find the penalty for a break of that length, and then apply the difference between that penalty and the previous one. Usually, that results in second-layoff penalties of between 10 and 50 points.

Measuring Return Aggression

In the last couple of years, I’ve gotten a lot of mileage out of a metric called Aggression Score (AS), first outlined here by Lowell West. The stat is so useful due to its simplicity. The more aggressive a player is, the more she’ll rack up both winners and unforced errors. AS, then, is essentially the rate at which a player hits winners and unforced errors.

Yet one limitation lies in Aggression Score’s simplicity. It works best when winners and unforced errors move together, and when they are roughly similar. If someone is having a really bad day, her unforced errors might skyrocket, resulting in a higher AS, even if the root cause of the errors is poor play, not aggression. On the flip side, a locked-in player will see her AS increase by hitting more winners, even if those winners are more a reflection of good form than a high-risk tactic.

I’ve long wanted to extend the idea behind Aggression Score to return tactics, but when we narrow our view to the second shot of the rally, the simplicity of the metric becomes a handicap. On the return, the vast majority of “aggressive” shots are errors, so the results will be swamped by error rate, minimizing the role of return winners, which are a more reliable indicator. Using Match Charting Project data from 2010-present women’s tennis, returns result in errors 18% of the time, while they turn into winners (or they induce forced errors) less than one-third as often, 5.5% of the time. The appealingly simple Aggression Score formula, narrowed to consider only returns of serve, won’t do the job here.

Return aggression score

Let’s walk through a formula to measure return aggression, using last month’s Miami final between Sloane Stephens and Jelena Ostapenko as an example. Tallying up return points (excluding aces and service winners), along with return errors* and return winners** for both players from the match chart, we get the following:

Returner          RetPts  RetErr  RetWin  RetE%  RetW%  
Sloane Stephens       64       9       1  14.1%   1.6%  
Jelena Ostapenko      63      11       6  17.5%   9.5%

* “errors” are a combination of forced and unforced, because most return errors are scored as forced errors, and because the distinction between the two is so unreliable as to be meaningless. Some forced error returns are nearly impossible to make, so they don’t really belong in this analysis, but with the state of available data, it’ll have to do.

** throughout this post, I’ll use “winners” as short-hand for “winners plus induced forced errors” — that is, shots that were good enough to end the point.

These numbers make clear which of the two players is the aggressive one, and they confirm the obvious: Ostapenko plays much higher-risk tennis than Stephens does. In this case, Ostapenko’s rates are nearly equal to or above the tour averages of 17.8% and 5.5%, while both of Stephens’s are well below them.

The next step is to normalize the error and winner rates so that we can more easily see how they relate to each other. To do that, I simply divide each number by the tour average:

Returner          RetE%  RetW%  RetE+  RetW+  
Sloane Stephens   14.1%   1.6%   0.79   0.28  
Jelena Ostapenko  17.5%   9.5%   0.98   1.73

The last two columns show the normalized figures, which reflect how each rate compares to tour average, where 1.0 is average, greater than 1 means more aggressive, and less than 1 means less aggressive.

We’re not quite done yet, because, as Ostapenko and Stephens illustrate, return winner rates are much noisier than return error rates. That’s largely a function of how few there are. The gap between the two players’ normalized rates, 0.28 and 1.73, looks huge, but represents a difference of only five winners. If we leave return winner rates untouched, we’ll end up with a metric that varies largely due to movement in winner rates–the opposite problem from where we started.

To put winners and errors on a more equal footing, we can express both in terms of standard deviations. The standard deviation of the adjusted error ratio is 0.404, while the standard deviation of the adjusted winner ratio is 0.768, so when we divide the ratios by the standard deviations, we’re essentially reducing the variance in the winner number by half. The resulting numbers tell us how many standard deviations a certain statistic is above or below the mean, and these final results give us winner and error rates that are finally comparable to each other:

Returner          RetE+  RetW+  RetE-SD  RetW-SD  
Sloane Stephens    0.79   0.28    -0.52    -0.93  
Jelena Ostapenko   0.98   1.73    -0.05     0.95

(Math-oriented readers might notice that the last two steps don’t need to be separate; we could just as easily think of these last two numbers as standard deviations above or below the mean of the original winner and error rates. I included the intermediate step to–I hope–make the process a bit more intuitive.)

Our final stat, Return Aggression Score (RAS) is simply the average of those two rates measured in standard deviations:

Returner          RetE-SD  RetW-SD    RAS  
Sloane Stephens     -0.52    -0.93  -0.73  
Jelena Ostapenko    -0.05     0.95   0.45

Positive numbers represent more aggression than tour average; negative numbers less aggression. Ostapenko’s +0.45 figure is higher than about 75% of player-matches among the nearly 4,000 in the Match Charting Project dataset, though as we’ll see, it is far more conservative than her typical strategy. Stephens’s -0.73 mark is at the opposite position on the spectrum, higher than only one-quarter of player-matches. It is also lower than her own average, though it is higher than the -0.97 RAS she posted in the US Open final last fall.

The extremes

The first test of any new metric is whether the results actually make sense, and we need look no further than the top ten most aggressive player-matches for confirmation. Five of the top ten most aggressive single-match return performances belong to Serena Williams, and the overall most aggressive match is Serena’s 2013 Roland Garros semifinal against Sara Errani, which rates at 3.63–well over three standard deviations above the mean. The other players represented in the top ten are Ostapenko, Oceane Dodin, Petra Kvitova, Madison Keys, and Julia Goerges–a who’s who of high-risk returning in women’s tennis.

The opposite end of the spectrum includes another group of predictable names, such as Simona Halep, Agnieszka Radwanska, Caroline Wozniacki, Annika Beck, and Errani. Two of Halep’s early matches are lowest and third-lowest, including the 2012 Brussels final against Radwanska, in which her return aggression was 1.6 standard deviations below the mean. It’s not as extreme a mark as Serena’s performances, but that’s the nature of the metric: Halep returned 46 of 48 non-ace serves, and none of the 46 returns went for winners. It’s tough to be less aggressive than that.

The leaderboard

The Match Charting Project has shot-by-shot data on at least ten matches each for over 100 WTA players. Of those, here are the top ten, as ranked by RAS:

Player                    Matches  RetPts   RAS  
Oceane Dodin                   11     665  1.18  
Aryna Sabalenka                11     816  1.12  
Camila Giorgi                  19    1155  1.07  
Mirjana Lucic                  11     707  1.05  
Julia Goerges                  27    1715  0.94  
Petra Kvitova                  65    4142  0.90  
Serena Williams                91    5593  0.90  
Jelena Ostapenko               35    2522  0.88  
Anastasia Pavlyuchenkova       21    1180  0.78  
Lucie Safarova                 34    2294  0.77

We’ve already seen some of these names, in our discussion of the highest single-match marks. When we average across contests, a few more players turn up with RAS marks over one full standard deviation above the mean: Aryna Sabalenka, Camila Giorgi, and Mirjana Lucic-Baroni.

Again, the more conservative players don’t look as extreme: Only Madison Brengle has a RAS more than one standard deviation below the mean. I’ve included the top 20 on this list because so many notable names (Wozniacki, Radwanska, Kerber) are between 11 and 20:

Player                Matches  RetPts     RAS  
Madison Brengle            11     702   -1.06  
Monica Niculescu           32    2099   -0.93  
Stefanie Voegele           12     855   -0.85  
Annika Beck                16    1181   -0.78  
Lara Arruabarrena          10     627   -0.72  
Johanna Larsson            14     873   -0.65  
Barbora Strycova           20    1275   -0.63  
Sara Errani                25    1546   -0.60  
Carla Suarez Navarro       36    2585   -0.55  
Svetlana Kuznetsova        27    2271   -0.55 

Player                Matches  RetPts     RAS  
Viktorija Golubic          16    1272   -0.53  
Agnieszka Radwanska        96    6239   -0.51  
Yulia Putintseva           22    1552   -0.51  
Caroline Wozniacki         80    5165   -0.50  
Christina McHale           11     763   -0.48  
Angelique Kerber           93    6611   -0.46  
Louisa Chirico             13     806   -0.44  
Darya Kasatkina            26    1586   -0.43  
Magdalena Rybarikova       12     725   -0.41  
Anastasija Sevastova       30    1952   -0.40

A few more notable names: Halep, Stephens and Elina Svitolina all count among the next ten lowest, with RAS figures between -0.30 and -0.36. The most “average” player among game’s best is Victoria Azarenka, who rates at -0.08. Venus Williams, Johanna Konta, and Garbine Muguruza make up a notable group of aggressive-but-not-really-aggressive women between +0.15 and +0.20, just outside of the game’s top third, while Maria Sharapova, at +0.63, misses our first list by only a few places.

Unsurprisingly, these results track quite closely to overall Aggression Score figures, as players who adopt a high-risk strategy overall are probably doing the same when facing the serve. This metric, however, allows to identify players–or even single matches–for which the two strategies don’t move in concert. Further, the approach I’ve taken here, to separate and normalize winners and errors, rather than treat them as an undifferentiated mass, could be applied to Aggression Score itself, or to other more targeted versions of the metric, such as a third-shot AS, or a backhand-specific AS.

As always, the more data we have, the more we can learn from it. Analyses like these are only possible with the work of the volunteers who have contributed to the Match Charting Project. Please help us continue to expand our coverage and give analysts the opportunity to look at shot-by-shot data, instead of just the basics published by tennis’s official federations.

Translating ATP Statistics Across Main Tour and Challenger Levels

Italian translation at settesei.it

What is the gap between the top-level ATP Tour and the lower-level ATP Challenger Tour? Some players pile up trophies in the minor leagues yet have a hard time converting that success to match wins on the big tour, while others struggle with the week-to-week grind of the challengers but excel when given opportunities on the larger stage.

Let’s take a look at a method that measures the difference between the skill level on the two tours. Once we can translate stats between levels, we can identify those players who are much better or worse than expected when they have the chance to compete against the best.

The algorithm I’ll use is almost identical to the one baseball analysts have used for decades to determine league equivalencies. For instance, we might find that a batting average of .300 in Triple-A (the highest minor league) is equivalent to .280 in the majors, meaning that, if a player is batting .300 in Triple-A, we’ll expect him to bat .280 in the majors. In tennis terms, it may be that a 10% ace rate in challengers is equivalent to a 8% ace rate on the main tour. Not every player will exhibit that precise drop in performance–some may even appear to get a little better–but on average, a league equivalency tells us what to expect when a player changes levels.

Here is the algorithm for league equivalencies, as applied to men’s tennis:

Pick a stat to focus on. I’ll use Total Points Won (TPW) here.
Neutralize that stat as much as possible. In baseball, that means controlling for the difference in parks; in tennis, it means controlling for competition. For the following, I’ve adjusted for each player’s quality of competition using a method I described about a year ago. Most players’ numbers are about the same after the adjustment, but a particularly easy or tough schedule means a bigger shift. For instance, Denis Shapovalov posted a TPW of 49.8% on the big tour last season, but because he played such high-quality competition, the adjustment bumps him up to 52.1%, 18th among tour regulars.
Identify players who competed at both levels, and find their adjusted stats at each level. Shapovalov played 18 tour-level matches and 30 challenger-level matches last year, with adjusted TPW numbers of 52.1% and 54.4%, respectively.
Calculate the ratio for each player. For Shapovalov last year, it was 1.044 (54.4 / 52.1).
Finally, take a weighted average of every player’s ratio. The weight is determined by the minimum number of matches played at either level, so for Shapovalov, it’s 18. Using the minimum means that a player like Gleb Sakharov (1 ATP match, 37 challenger matches) can be included in the calculation, but has very little effect on the end result.

Here are the results for the last six full seasons. Each ratio is the relationship between challenger-level TPW and tour-level TPW:

Year  Ratio  
2017  1.086  
2016  1.086  
2015  1.098  
2014  1.103  
2013  1.100  
2012  1.100

The average of these yearly equivalency factors is roughly the difference between a 52.5% TPW at challengers and a 48.0% TPW on the main tour. The shift from 2012-15 to 2016-17 may reflect the injuries that have sidelined the elites. With fewer elite players on court, the gap between the two tours narrows.

Now that we know the difference between the levels, we can find the players who defy the usual patterns. Of the 100 players with the most “paired” matches–that is, with the most matches at both levels in the same years–here are the 20 with the lowest ratios. Low ratios mean less difference in performance between the two levels, so these guys are either overperforming at tour level or underperforming at challengers:

Player              ATP M  CH M  Min M  Ratio  
Matthew Ebden          62   140     39  0.982  
Jared Donaldson        68    78     37  1.030  
Jack Sock              81    45     38  1.039  
James Duckworth        53   156     53  1.042  
Andrey Rublev          56    79     42  1.047  
Vasek Pospisil         96    76     60  1.047  
Thiemo De Bakker       48    87     44  1.048  
Samuel Groth           84   133     58  1.049  
Michael Berrer         59   107     56  1.050  
Ruben Bemelmans        41   178     41  1.052  
Dustin Brown          120   173    111  1.055  
Benoit Paire          295    53     53  1.059  
Peter Gojowczyk        46   132     44  1.059  
Michael Russell        58    78     58  1.061  
Marius Copil           58   180     58  1.063  
Taylor Harry Fritz     59    44     41  1.065  
Jordan Thompson        38    88     38  1.066  
Illya Marchenko        56   116     37  1.066  
Tatsuma Ito            65   179     65  1.066  
Ryan Harrison         124    84     59  1.068

The middle columns show the total number of ATP matches, challenger matches, and “paired” matches between 2012 and 2017 (“Min M”) for each player. (The last number gives an indication of just how much data was available for the single-player calculation.) Aside from a few big-serving North Americans near the top of this list, I don’t see a lot of obvious commonalities. There are some youngsters, some veterans, more big servers than not, but nothing obvious.

(Shapovalov doesn’t have enough paired matches to qualify, but his overall ratio is 1.035, good for third on this list.)

Here is the opposite list, the quintile of 20 players who have overperformed at challengers or underperformed on tour:

Player               ATP M  CH M  Min M  Ratio  
Florian Mayer          152    45     45  1.180  
Mikhail Youzhny         91    38     38  1.169  
Aljaz Bedene           144   121     80  1.160  
Filippo Volandri        62   101     62  1.158  
Robin Haase            194    71     71  1.157  
Tobias Kamke           102   144     73  1.155  
Adrian Mannarino       234   115     86  1.155  
Filip Krajinovic        36   167     36  1.148  
Albert Ramos           111    67     62  1.144  
Paul Henri Mathieu     147    96     82  1.141  
Kenny De Schepper       77   196     77  1.140  
Facundo Bagnis          45   197     45  1.136  
Pablo Cuevas           127    52     43  1.136  
Ivan Dodig              76    48     41  1.135  
Santiago Giraldo       146    70     56  1.135  
Paolo Lorenzi          204   191    124  1.135  
Thomaz Bellucci        162    44     44  1.134  
Albert Montanes        113   109     70  1.130  
Rogerio Dutra Silva     57   210     57  1.130  
Lukas Lacko            122   181    108  1.129

There are more clay-courters here than on the first list, and the very top of the ranking includes veterans who have mastered the challenger level, even if they still struggle to maintain a foothold on the main tour. I’ve had to exclude one player who belongs on this list: Gilles Muller broke my algorithm with his 45-9 challenger season in 2014. When I took him out of the 2014 calculations, the overall numbers changed very little, but it means no Muller here. Whatever his exact ratio, I can say that his tour-level performance hasn’t matched that 2014 run at challengers.

The bottoms of the two lists indicate that there isn’t that much variation between players. The middle 60% of players all have ratios between about 1.07 and 1.13, while the yearly averages hover around 1.09 and 1.10. Some players under consideration here have fewer than 50 “paired” matches over the six seasons, so a difference of a couple hundredths is far too little to draw any conclusions.

This algorithm, beyond suggesting what to expect from players when they move up from challengers to the main tour, could apply the same reasoning to other pairs of levels, such as ITF Futures and challengers, or women’s ITFs and the WTA tour. It could even compare narrower levels, such as ITF $10,000 events with ITF $15,000s, or ATP 250s with ATP 500s. The method is a staple of analytics in other sports, and it has a place in tennis, as well.