Category: Win probability

Smaller Swings In Big Moments

Italian translation at settesei.it

Despite the name, unforced errors aren’t necessarily bad. Sometimes, the right tactic is to play more aggressively, and in order to hit more winners, most players will commit more errors as well. Against some opponents, increasing the unforced error count–as long as there is a parallel improvement in winners or other positive point-ending shots–might be the only way to win.

Last week, I showed that one of the causes of Angelique Kerber’s first-round loss was her disproportionate number of errors in big moments. But as my podcasting partner Carl Bialik pointed out, that isn’t the whole story. If Kerber played more aggressively on the most important points–one possible cause of more errors–it might be the case that her winner rate was higher, as well. Since the 6-2 6-2 scoreline was so heavily tilted against her, it was a safe bet that Kerber recorded more high-leverage errors than winners. Still, Carl makes a valid point, and one worth testing.

To do so, let’s revisit the data: 500 women’s singles matches from the last four majors and the first four rounds of this year’s French Open. By measuring the importance of each point, we can determine the average leverage (LEV) of every point in each match, along with the average leverage of points which ended with a player hitting an unforced error, or a winner. Last week, we found that Kerber’s UEs in her first-round loss had an average LEV of 5.5%, compared to a LEV of 3.8% on all other points. For today’s purposes, let’s use match averages as a reference point: Her average UE LEV of 5.5% also compares unfavorably to the overall match average LEV of 4.1%.

What about winners? Kerber’s 15 winners came on points with an average LEV of 3.9%, below the match average. Case closed: On more important points, Kerber was more likely to commit an error, and less likely to hit a winner.

Across the whole population, players hit more errors and fewer winners in crucial moments, but only slightly. Points ending in errors are about one percent more important than average (percent, not percentage point, so 4.14% instead of 4.1%), and points ending in winners are about two percent less important than average. In bigger moments, players increase their winner rate about 39% of the time, and they improve their W-UE ratio about 45% of the time. Point being, there are tour-wide effects on more important points, but they are quite small.

Of course, Kerber’s first-round upset isn’t indicative of how she has played at Slams in general. In my article last week, I mentioned the four players who did the best job of reducing errors at big moments: Kerber, Agnieszka Radwanska, Timea Bacsinszky, and Kiki Bertens. Kerber and Radwanska both hit fewer winners on big points as well, but Bacsinszky and Bertens manage a perfect combination, hitting slightly more winners as the pressure cranks up. Among players with more than 10 Slam matches since last year’s French, Bacsinszky is the only one to hit winners on more important points than her unforced errors over 75% of the time.

Compared to her peers, Kerber’s big-moment tactics are remarkably passive. The following table shows the 21 women for whom I have data on at least 13 matches. “UE Rt.” (“UE Ratio”) is similar to the metric I used last week, comparing the average importance of points ending in errors to average points; “W Ratio” is the same, but for points ending in winners, and “W+UE Ratio” is–you guessed it–a (weighted) combination of the two. The combined measure serves as an rough approximation of aggression on big points, where ratios below 1 are more passive than the player’s typical tactics and ratios above 1 are more aggressive.

Player                     M  UE Rt.  W Rt.  W+UE Rt.  
Angelique Kerber          20    0.92   0.85      0.88  
Alize Cornet              13    0.92   0.87      0.94  
Agnieszka Radwanska       17    0.91   0.95      0.95  
Simona Halep              19    0.93   0.94      0.95  
Samantha Stosur           13    0.95   0.98      0.96  
Timea Bacsinszky          14    0.89   1.02      0.97  
Elina Svitolina           15    1.02   0.95      0.97  
Karolina Pliskova         18    0.97   0.98      0.97  
Caroline Wozniacki        14    0.93   1.00      0.97  
Johanna Konta             13    1.00   0.97      0.98  
Caroline Garcia           14    0.94   1.02      0.98  
Svetlana Kuznetsova       17    0.96   0.98      0.99  
Garbine Muguruza          20    1.02   0.94      0.99  
Venus Williams            25    1.00   0.97      0.99  
Elena Vesnina             13    0.96   1.03      0.99  
Anastasia Pavlyuchenkova  15    1.03   0.99      0.99  
Coco Vandeweghe           13    1.08   0.95      1.01  
Madison Keys              13    1.01   1.02      1.01  
Serena Williams           27    0.99   1.05      1.02  
Carla Suarez Navarro      14    1.00   1.14      1.05  
Dominika Cibulkova        14    1.11   1.03      1.07

Kerber’s combined measure stands out from the pack. Her point-ending shots–both winners and errors, but especially winners–occur disproportionately on less important points, and the overall effect is double that of the next most passive big-moment player, Alize Cornet. Every other player is close enough to neutral that I would hesitate before making any conclusions about their pressure-point tactics.

Even when Kerber wins, she does so with effective defense at key points. In only two of her last 20 matches at majors did her winners occur on particularly important points. (Incidentally, one of those two was last year’s US Open final.) In general, her brand of passivity works–she won 16 of those matches. But defensive play doesn’t leave very much room for error–figuratively or literally. The tactics were familiar and proven, but against Makarova, they were poorly executed.

Angelique Kerber’s Unclutch Unforced Errors

Italian translation at settesei.it

It’s been a rough year for Angelique Kerber. Despite her No. 1 WTA ranking and place at the top of the French Open draw, she lost her opening match on Sunday against the unseeded Ekaterina Makarova. Adding insult to injury, the loss goes down in the record books as a lopsided-looking 6-2 6-2.

Andrea Petkovic chimed in with her diagnosis of Kerber’s woes:

She’s simply playing without confidence right now. It was tight, even though the scoreline was 2 and 2 but everyone who knows a thing about tennis knew that Angie made errors whenever it mattered because she’s playing without any confidence right now – errors she didn’t make last year.

This is one version of a common analysis: A player lost because she crumbled on the big points. While that probably doesn’t cover all of Kerber’s issues on Sunday–Makarova won 72 points to her 55–it is true that big points have a disproportionate effect on the end result. For every player who squanders a dozen break points yet still wins the match, there are others who falter at crucial moments and ultimately lose.

This family of theories–that a player over- or under-performed at big moments–is testable. For instance, I showed last summer that Roger Federer’s Wimbledon loss to Milos Raonic was due in part to his weaker performance on more important points. We can do the same with Kerber’s early exit.

Here’s how it works. Once we calculate each player’s probability of winning the match before each point, we can assign each point a measure of importance–I prefer to call it leverage, or LEV–that quantifies how much the single point could effect the outcome of the match. At 3-0, 40-0, it’s almost zero. At 3-3, 40-AD in the deciding set, it might be over 10%. Across an entire tournament’s worth of matches, the average LEV is around 5% to 6%.

If Petko is right, we’ll find that the average LEV of Kerber’s unforced errors was higher than on other points. (I’ve excluded points that ended with the serve, since neither player had a chance to commit an unforced error.) Sure enough, Kerber’s 13 groundstroke UEs (that is, excluding double faults) had an average LEV of 5.5%, compared to 3.8% on points that ended some other way. Her UE points were 45% more important than non-UE points.

Let’s put that number in perspective. Among the 86 women for whom I have point-by-point UE data for their first-round matches this week*, ten timed their errors even worse than Kerber did. Magdalena Rybarikova was the most extreme: Her eight UEs against Coco Vandeweghe were more than twice as important, on average, as the rest of the points in that match. Seven of the ten women with bad timing lost their matches, and two others–Agnieszka Radwanska and Marketa Vondrousova–committed so few errors (3 and 4, respectively), that it didn’t really matter. Only Dominika Cibulkova, whose 15 errors were about as badly timed as Kerber’s, suffered from unclutch UEs yet managed to advance.

* This data comes from the Roland Garros website. I aggregate it after each major and make it available here.

Another important reference point: Unforced errors are evenly distributed across all leverage levels. Our instincts might tell us otherwise–we might disproportionately recall UEs that came under pressure—-but the numbers don’t bear it out. Thus, Kerber’s badly timed errors are just as badly timed when we compare her to tour average.

They are also poorly timed when compared to her other recent performances at majors. Petkovic implied as much when she said her compatriot was making “errors she didn’t make last year.” Across her 19 matches at the previous four Slams, her UEs occurred on points that were 11% less important than non-UE points. Her errors caused her to lose relatively more important points in only 5 of the 19 matches, and even in those matches, the ratio of UE leverage to non-UE leverage never exceeded 31%, her ratio in Melbourne this year against Tsurenko. That’s still better than her performance on Sunday.

Across so many matches, a difference of 11% is substantial. Of the 30 players with point-by-point UE data for at least eight matches at the previous four majors, only three did a better job timing their unforced errors. Radwanska heads the list, at 16%, followed by Timea Bacsinszky at 14% and Kiki Bertens at 12%. The other 26 players committed their unforced errors at more important moments than Kerber did.

As is so often the case in tennis, it’s difficult to establish if a stat like this is indicative of a longer-trend trend, or if it is mostly noise. We don’t have point-by-point data for most of Kerber’s matches, so we can’t take the obvious next step of checking the rest of her 2017 matches for similarly unclutch performances. Instead, we’ll have to keep tabs on how well she limits UEs at big moments on those occasions where we have the data necessary to do so.

New at Tennis Abstract: Point-by-Point Stats

Yesterday, I announced the new ATP doubles results on Tennis Abstract. Today, I want to show you something else I rolled out over the offseason: sequential point-by-point stats for more than 100,000 matches.

Traditional match stats can do no more than summarize the action. Point-by-point stats are so much more revealing: They show us how matches unfold and allow us to look much deeper into topics such as momentum and situational skill. These are subjects that remain mysteries–or, at the very least, poorly quantified–in tennis.

As an example, let’s take a look at the new data available for one memorable contest, the World Tour Finals semifinal between Andy Murray and Milos Raonic:

The centerpiece of each page is a win probability graph, which shows the odds that one player would win the match after each point. These graphs do not take player skill into account, though they are adjusted for gender and surface. The red line shows one player’s win probability, while the grey line indicates “volatility”–a measure of how much each point matters. You can see exact win probability and volatility numbers by moving your cursor over the graph. Most match graphs aren’t nearly as dramatic as this one; of course, most matches aren’t nearly as dramatic as this one was.

(I’ve written a lot about win probability in the past, and I’ve also published the code I use to calculate in-match win probability.)

Next comes a table with situational serving stats for both players. In the screenshot above, you can see deuce/ad splits; the page continues, with tiebreak-specific totals and tallies for break points, set points, and match points. After that is an exhaustive, point-by-point text recap of the match, which displays the sequence of every point played.

I’ve tried to make these point-by-point match pages as easy to find as possible. Whenever you see a link on a match score, just click that link for the point-by-point page. For instance, here is part of Andy Murray’s page, showing where to click to find the Murray-Raonic example shown above:

As you can see from all the blue scores in this screenshot, most 2016 ATP tour-level matches have point-by-point data available. The same is true for the last few seasons, as well as top-level WTA matches. The lower the level, the fewer matches are available, but you might be surprised by the breadth and depth of the coverage. The site now contains point-by-point data for almost half of 2016 main-draw men’s Futures matches. For instance, here’s the graph for a Futures final last May between Stefanos Tsitsipas and Casper Ruud.

I’ll keep these as up-to-date as I can, but with my current setup, you can expect to wait 1-4 weeks after a match before the point-by-point page becomes available. I’m hoping to further automate the process and shorten the wait over the course of this season.

Enjoy!

The Most Exciting Matches of the 2016 WTA Season

Italian translation at settesei.it

In my most recent piece for The Economist, I used a metric called Excitement Index (EI) to consider the implications of shortening singles matches to a format like the no-ad, super-tiebreak rules used for doubles. In my simulations, the shorter format didn’t fare well: The most gripping contests are often the longest ones, and the full-length third set is frequently the best part.

I used data from ATP tournaments in that piece, and several readers have asked how women’s matches score on the EI scale. Many matches from the 2016 season rate extremely highly, while some players we tend to think of as exciting fail to register among the best by this metric. I’ll share some of the results in a moment.

First, a quick overview of EI. We can calculate the probability that each player will win a match at any point in the contest, and using those numbers, it’s possible to determine the leverage of every point–that is, the difference between a player’s odds if she wins the next point and her odds if she loses it. At 40-0, down a break in the first set, that leverage is very low: less than 2%. In a tight third-set tiebreak, leverage can climb as high as 25%. The average point is around 5% to 6%, and as long as neither player has a substantial lead, points at 30-30 or later are higher.

EI is calculated by averaging the leverage of every point in the match. The more high-leverage points, the higher the EI. To make the results a bit more viewer-friendly, I multiply the average leverage by 1,000, so if the typical point has the potential for a 5% (0.05) swing, the EI is 50. The most boring matches, like Garbine Muguruza‘s 6-1 6-0 dismantling of Ekaterina Makarova in Rome, rate below 25. The most exciting will occasionally top 100, and the average WTA match this year scored a 53.7. By comparison, the average ATP match this year rated at 48.9.

Of course, the number and magnitude of crucial moments isn’t the only thing that can make a tennis match “exciting.” Finals tend to be more gripping than first-round tilts, long rallies and daring net play are more watchable than error-riddled ballbashing, and Fed Cup rubbers feature crowds that can make the warmup feel like a third-set tiebreak. When news outlets make their “Best Matches of 2016” lists, they’ll surely take some of those other factors into account. EI takes a narrower view, and it is able to show us which matches, independent of context, offered the most pressure-packed tennis.

Here are the top ten matches of the 2016 WTA season, ranked by EI:

Tournament    Match                Score                    EI  
Charleston    Lucic/Mladenovic     4-6 6-4 7-6(13)       109.9  
Wimbledon     Cibulkova/Radwanska  6-3 5-7 9-7           105.0  
Wimbledon     Safarova/Cepelova    4-6 6-1 12-10         101.7  
Kuala Lumpur  Nara/Hantuchova      6-4 6-7(4) 7-6(10)    100.2  
Brisbane      CSN/Lepchenko        4-6 6-4 7-5            99.0  
Quebec City   Vickery/Tig          7-6(5) 6-7(3) 7-6(7)   98.5  
Miami         Garcia/Petkovic      7-6(5) 3-6 7-6(2)      98.1  
Wimbledon     Vesnina/Makarova     5-7 6-1 9-7            97.2  
Beijing       Keys/Kvitova         6-3 6-7(2) 7-6(5)      96.8  
Acapulco      Stephens/Cibulkova   6-4 4-6 7-6(5)         96.7

Getting to 6-6 in the final set is clearly a good way to appear on this list. The top fifty matches of the season (out of about 2,700) all reached at least 5-5 in the third. The highest-rated clash that didn’t get that far was Angelique Kerber‘s 1-6 7-6(2) 6-4 defeat of Elina Svitolina, with an EI of 88.2. Svitolina’s 4-6 6-3 6-4 victory over Bethanie Mattek Sands in Wuhan, the top match on the list without any sets reaching 5-5, scored an EI of 87.3.

Wimbledon featured an unusual number of very exciting matches this year, especially compared to Roland Garros and the Australian Open, the other tournaments that forgo a tiebreak in the final set. The top-rated French Open contest was the first-rounder between Johanna Larsson and Magda Linette, which scored 95.3 and ranks 13th for the season, while the highest EI among Aussie Open matches is all the way down at 27th on the list, a 92.8 between Monica Puig and Kristyna Pliskova.

Dominika Cibulkova is the only player who appears twice on this list. That doesn’t mean she’s a sure thing for exciting matches: As we’ll see, elite players rarely are. The only year-end top-tenner who ranks among the highest average EIs is Svetlana Kuznetsova, who played as many “very exciting” matches–those rating among the top fifth of matches this season–as any other woman on tour:

Rank  Player                M  Avg EI  V. Exc  Exc %  Bor %  
1     Kristina Mladenovic  60    59.8      19  55.0%  25.0%  
2     Christina McHale     46    59.6      16  50.0%  19.6%  
3     Heather Watson       35    58.5      12  48.6%  25.7%  
4     Jelena Jankovic      43    57.6      12  55.8%  30.2%  
5     Svetlana Kuznetsova  64    57.4      21  48.4%  32.8%  
6     Venus Williams       38    57.1      10  55.3%  31.6%  
7     Yanina Wickmayer     43    56.5      13  46.5%  30.2%  
8     Alison Riske         46    56.5      10  45.7%  32.6%  
9     Caroline Garcia      62    56.4      18  43.5%  33.9%  
10    Irina-Camelia Begu   42    56.4      14  45.2%  40.5%

(Minimum 35 tour-level matches (“M” above), excluding retirements. My data is also missing a random handful of matches throughout the season.)

The “V. Exc” column tallies how many top-quintile matches the player took part in. The “Exc %” column shows the percent of matches that rated in the top 40% of all WTA contests, while “Bor %” shows the same for the bottom 40%, the more boring matches. Big servers who reach a disproportionate number of tiebreaks and 7-5 sets do well on this list, though it is far from a perfect correspondence. Tiebreaks can create a lot of big moments, but if there were many love service games en route to 6-6, the overall picture isn’t nearly so exciting.

Unlike Kuznetsova, who played a whopping 32 deciding sets this year, most of the other top women enjoy plenty of blowouts. Muguruza, Simona Halep, and Serena Williams occupy the very last three places on the average-EI ranking, largely because when they win, they do so handily–and they win a lot. The next table shows the WTA year-end top-ten, with their ranking (out of 59) on the average-EI list:

Rank  Player        WTA#  Matches  Avg EI  V. Exc  Exc %  Bor %  
5     Kuznetsova       9       64    57.4      21  48.4%  32.8%  
13    Pliskova         6       66    55.6      19  48.5%  39.4%  
16    Keys             8       64    55.4      13  40.6%  35.9%  
23    Cibulkova        5       68    54.6      21  42.6%  42.6%  
28    Kerber           1       77    54.0      12  42.9%  41.6%  
      tour average                   53.7          40.0%  40.0%  
41    Radwanska        3       69    52.5      12  29.0%  44.9%  
51    Konta           10       67    51.2      12  34.3%  46.3%  
57    Muguruza         7       51    49.9       5  33.3%  43.1%  
58    Halep            4       59    49.6       8  30.5%  50.8%  
59    Williams         2       44    48.1       3  27.3%  50.0%

It’s a good thing that fans love Serena, because her matches rarely provide much in the way of big moments. As low as Williams and Halep rate on this measure, Victoria Azarenka scores even lower. Her Miami fourth-rounder against Muguruza was her only match this season to rank in the “exciting” category, and her average EI was a mere 44.0.

Clearly, EI isn’t much of a method for identifying the best players. Even looking at the lowest-rated competitors by EI would be misleading: In 56th place, right above Muguruza, is the otherwise unheralded Nao Hibino. EI excels as a metric for ferreting out the most riveting individual matches, whether they were broadcast worldwide or ignored entirely. And the next time someone suggests shortening matches, EI is a great tool to highlight just how much excitement would be lost by doing so.

Measuring the Clutchness of Everything

Italian translation at settesei.it

Matches are often won or lost by a player’s performance on “big points.” With a few clutch aces or un-clutch errors, it’s easy to gain a reputation as a mental giant or a choker.

Aside from the traditional break point stats, which have plenty of limitations, we don’t have a good way to measure clutch performance in tennis. There’s a lot more to this issue than counting break points won and lost, and it turns out that a lot of the work necessary to quantify clutchness is already done.

I’ve written many times about win probability in tennis. At any given point score, we can calculate the likelihood that each player will go on to win the match. Back in 2010, I borrowed a page from baseball analysts and introduced the concept of volatility, as well. (Click the link to see a visual representation of both metrics for an entire match.) Volatility, or leverage, measures the importance of each point–the difference in win probability between a player winning it or losing it.

To put it simply, the higher the leverage of a point, the more valuable it is to win. “High leverage point” is just a more technical way of saying “big point.”  To be considered clutch, a player should be winning more high-leverage points than low-leverage points. You don’t have to win a disproportionate number of high-leverage points to be a very good player–Roger Federer’s break point record is proof of that–but high-leverage points are key to being a clutch player.

(I’m not the only person to think about these issues. Stephanie wrote about this topic in December and calculated a full-year clutch metric for the 2015 ATP season.)

To make this more concrete, I calculated win probability and leverage (LEV) for every point in the Wimbledon semifinal between Federer and Milos Raonic. For the first point of the match, LEV = 2.2%. Raonic could boost his match odds to 50.7% by winning it or drop to 48.5% by losing it. The highest leverage in the match was a whopping 32.8%, when Federer (twice) had game point at 1-2 in the fifth set. The lowest leverage of the match was a mere 0.03%, when Raonic served at 40-0, down a break in the third set. The average LEV in the match was 5.7%, a rather high figure befitting such a tight match.

On average, the 166 points that Raonic won were slightly more important, with LEV = 5.85%, than Federer’s 160, at LEV = 5.62%. Without doing a lot more work with match-level leverage figures, I don’t know whether that’s a terribly meaningful difference. What is clear, though, is that certain parts of Federer’s game fell apart when he needed them most.

By Wimbledon’s official count, Federer committed nine unforced errors, not counting his five double faults, which we’ll get to in a minute. (The Match Charting Project log says Fed had 15, but that’s a discussion for another day.) There were 180 points in the match where the return was put in play, with an average LEV = 6.0%. Federer’s unforced errors, by contrast, had an average LEV nearly twice as high, at 11.0%! The typical leverage of Raonic’s unforced errors was a much less noteworthy 6.8%.

Fed’s double fault timing was even worse. Those of us who watched the fourth set don’t need a fancy metric to tell us that, but I’ll do it anyway. His five double faults had an average LEV of 13.7%. Raonic double faulted more than twice as often, but the average LEV of those points, 4.0%, means that his 11 doubles had less of an impact on the outcome of the match than Roger’s five.

Even the famous Federer forehand looks like less of a weapon when we add leverage to the mix. Fed hit 26 forehand winners, in points with average LEV = 5.1%. Raonic’s 23 forehand winners occurred during points with average LEV = 7.0%.

Taking these three stats together, it seems like Federer saved his greatness for the points that didn’t matter as much.

The bigger picture

When we look at a handful of stats from a single match, we’re not improving much on a commentator who vaguely summarizes a performance by saying that a player didn’t win enough of the big points. While it’s nice to attach concrete numbers to these things, the numbers are only worth so much without more context.

In order to gain a more meaningful understanding of this (or any) performance with leverage stats, there are many, many more questions we should be able to answer. Were Federer’s high-leverage performances typical? Does Milos often double fault on less important points? Do higher-leverage points usually result in more returns in play? How much can leverage explain the outcome of very close matches?

These questions (and dozens, if not hundreds more) signal to me that this is a fruitful field for further study. The smaller-scale numbers, like the average leverage of points ending with unforced errors, seem to have particular potential. For instance, it may be that Federer is less likely to go for a big forehand on a high-leverage point.

Despite the dangers of small samples, these metrics allow us to pinpoint what, exactly, players did at more crucial moments. Unlike some of the more simplistic stats that tennis fans are forced to rely on, leverage numbers could help us understand the situational tendencies of every player on tour, leading to a better grasp of each match as it happens.

How Dangerous Is It To Fix a Single Service Game?

Italian translation at settesei.it

Earlier this week, I offered a rough outline of the economics of fixing tennis matches, calculating the expected prize money that players forgo at various levels when they lose on purpose. The vast gulf between prize money, especially at lower-level events, and fixing fees suggests that gamblers must pay high premiums to convince players to do something ethically repugnant and fraught with risk.

So much for match-level fixes. What about single service games? In Ben Rothenberg’s recent report, a shadowy insider offers the following data points:

Buying a service break at a Futures event cost $300 to $500, he said. A set was $1,000 to $2,000, and a match was $2,000 to $3,000.

In other words, a service break is valued at between 10% and 25% the cost of an entire match. The article doesn’t mention service-break prices at higher levels, so we’ll have to use the Futures numbers as our reference point.

Selling a service break might be a way to have your cake and eat it too, taking some cash from gamblers while retaining the chance to advance in the draw and earn ranking points. But it won’t always work out that way.

I ran some simulations to see how much a service break should cost, based on the simplifying assumption that prices correspond to chances of winning and, by extension, forgone prize money. It turns out that the range of 10% to 25% is exactly right.

Let’s start with the simplest scenario: Two equal men with middle-of-the-road serves, which win them 63% of service points. In an honest match, these two would each have a 50% chance of winning. If one of them guarantees a break in his second service game, he is effectively lowering his chances of winning the match to 38.5%. dropping his expected prize money for the tournament by 23%.

If our players have weaker serves, for instance each winning 55% of service points, the fixer’s chances of winning the match fall to about 42%, only a 16% haircut. With stronger serves, using the extreme case of 70% of points going the way of the server, the fixer’s chances drop to 34%, a loss of 32% in his expected prize money.

This last scenario–two equal players with big serves–is the one that confers the most value on a single service break. We can use that 32% sacrifice as an upper bound for the worth of a single fixed break.

Fixed contests have more value to gamblers when the better player is guaranteed to lose, and in those cases, a service break doesn’t have as much impact on the outcome of the match. If the fixer is considerably better than his opponent, he was probably going to break serve a few times more than his opponent would, so losing a single game is less likely to determine the outcome of the match.

Let’s take a few examples:

  • If one player wins 64% of service points and other wins 62%, the favorite has a 60% chance of winning. If he fixes one service break, his chances of winning fall to just below 48%, about a 20% drop in expected prize money.
  • When one player wins 65% of service points against an opponent winning 61%, his chances in an honest match are 69.3%. Giving up one fixed service break, his odds fall to 57.4%, a sacrifice of roughly 17%.
  • A 67% server facing a 60% server has an 80.8% chance of winning. With one fixed service break, that drops to 70.7%, a loss of 12.5%.
  • A huge favorite winning 68% of service points against his opponent’s 58% has an 89.5% chance of advancing to the next round. Guarantee a break in one of his service games, and his odds drop to 82%, a loss of 8.4%.

With the exception of very lopsided matches (for which there might not be as many betting markets), we have our lower bound, not far below 10%.

The average Futures first-rounder, if we can generalize from such a mixed bag of matches, is somewhere in the middle of those examples–not an even contest, but without a heavy favorite. So the typical value of a fixed service break is between about 12% and 20% of the value of the match, right in the middle of the range of estimates given by Rothenberg’s source.

Even in this hidden, illegal marketplace, the numbers we’ve seen so far suggest that both gamblers and players act reasonably rationally. Amid a sea of bad news, that’s a good sign for tennis’s governing bodies: It promises that players will respond in a predictable manner to changing incentives. Unfortunately, it remains to be seen whether the incentives will change.

The Dreaded Deficit at the Tiebreak Change of Ends

Italian translation at settesei.it

Some of tennis’s conventional wisdom manages to be both blindingly self-evident and obviously wrong. Give pundits a basic fact (winning more points is good), add a dash of perceived momentum, and the results can be toxic.

A great example is the tiebreak change of ends. The typical scenario goes something like this: Serving at 2-3 in a tiebreak, a player loses a point on serve, going down a minibreak to 2-4. As the players change sides, a commentator says, “You really don’t want to go into this change of ends without at least keeping the score even.”

While the full rationale is rarely spelled out, the implication is that losing that one point–going from 2-3 to 2-4–is somehow worse than usual because the point precedes the changeover. Like the belief that the seventh game of the set is particularly important, this has passed, untested, into the canon.

Let’s start with the “blindingly self-evident” part. Yes, it’s better to head into the change of ends at 3-3 than it is at 2-4. In a tiebreak, every point is crucial. Based on a theoretical model and using sample players who each win 65% of service points, here are the odds of winning a tiebreak from various scores at the changeover:

Score  p(Win)  
1*-5     5.4%  
2*-4    21.5%  
3*-3    50.0%  
4*-2    78.5%  
5*-1    94.6%

It’s easy to sum that up: You really want to win that sixth point. (Or, at least, several of the points before the sixth.) On the other hand, compare that to the scenarios after eight points:

Score  p(Win)  
2*-6     2.6%  
3*-5    17.6%  
4*-4    50.0%  
5*-3    82.4%  
6*-2    97.4%

At the risk of belaboring the obvious, when the score is close, points become more important later in the tiebreak. The outcome at 4-4 matters more than at 3-3, which matters more than at 2-2, and so on. If players changed ends after eight points, we’d probably bestow some magical power on that score instead.

Real-life outcomes

So far, I’ve only discussed what the model tells us about win probabilities at various tiebreak scores. If the pundits are right, we should see a gap between the theoretical likelihood of winning a tiebreak from 2-4 and the number of times that players really do win tiebreaks from those scores. The model says that players should win 21.5% of tiebreaks from 2*-4; if the conventional wisdom is correct, we would find that players win even fewer tiebreaks when trying to come back from that deficit.

By analyzing the 20,000-plus tiebreaks in this dataset, we find that the opposite is true. Falling to 2-4 is hugely worse than reaching the change of ends at 3-3, but it isn’t worse than the model predicts–it’s a bit better.

To quantify the effect, I determined the likelihood that the player serving immediately after the changeover would win the tiebreak, based on each player’s service points won throughout the match and the model I’ve referred to above. By aggregating all of those predictions, together with the observed result of each tiebreak, we can see how real life compares to the model.

In this set of tiebreaks, a player serving at 2-4 would be expected to win 20.9% of the time. In fact, these players go to win the tiebreak 22.0% of the time–a small but meaningful difference. We see an even bigger gap for players returning at 2-4. The model predicts that they would win 19.9% of the time, but they end up winning 22.1% of these tiebreaks.

In other words, after six points, the player with more points is heavily favored, but if there’s any momentum–that is, if either player has more of an advantage than the mere score would suggest–the edge belongs the player trailing in the tiebreak.

Sure enough, we see the same effect after eight points. Serving at 3-5, players in this dataset have a 16.3% (theoretical) probability of winning the tiebreak, but they win 19.0% of the time. Returning at 3-5, their paper chance is 17.2%, and they win 19.5%.

There’s nothing special about the first change of ends, and there probably isn’t any other point in a tiebreak that is more crucial than the model suggests. Instead, we’ve discovered that underdogs have a slightly better chance of coming back than their paper probabilities indicate. I suspect we’re seeing the effect of front-runners getting tight and underdogs swinging more freely–an aspect of tennis’s conventional wisdom that has much more to recommend itself than the idea of a magic score after the first six points of a tiebreak.

The Pivotal Point of 15-30

According to nearly every tennis commentator I’ve ever heard, 15-30 is a crucial point, especially in men’s tennis, where breaks of serve are particularly rare. One reasonable explanation I’ve heard is that, from 15-30, if the server loses either of the next two points, he’ll face break point.

Another way of looking at it is with a theoretical model. A player who wins 65% of service points (roughly average on the ATP tour) has a 62% chance of winning the game from 15-30. If he wins the next point, the probability rises to 78% at 30-all, but if he loses the next point, he will only have a 33% chance of saving the game from 15-40.

Either way, 15-30 points have a lot riding on them. In line with my analysis of the first point of each game earlier this week, let’s take a closer look at 15-30 points–the odds of getting there, the outcome of the next point, and the chances of digging out a hold, along with a look at which players are particularly good or bad in these situations.

Reaching 15-30

In general, 15-30 points come up about once every four games, and no more or less often than we’d expect. In other words, games aren’t particularly likely or unlikely to reach that score.

On the other hand, some particular players are quite a bit more or less likely.  Oddly enough, big servers show up at both extremes. John Isner is the player who–relative to expectations–ends up serving at 15-30 the most often: 13% more than he should. Given the very high rate at which he wins service points, he should get to 15-30 in only 17% of service games, but he actually reaches 15-30 in 19% of service games.

The list of players who serve at 15-30 more often than they should is a very mixed crew. I’ve extended this list to the top 13 in order to include another player in Isner’s category:

Player                 Games  ExpW  ActW  Ratio  
John Isner             3166    537   608   1.13  
Joao Sousa             1390    384   432   1.12  
Janko Tipsarevic       1984    444   486   1.09  
Tommy Haas             1645    368   401   1.09  
Lleyton Hewitt         1442    391   425   1.09  
Tomas Berdych          3947    824   894   1.08  
Vasek Pospisil         1541    361   390   1.08  
Rafael Nadal           3209    661   713   1.08  
Pablo Andujar          1922    563   605   1.08  
Philipp Kohlschreiber  2948    652   698   1.07  
Gael Monfils           2319    547   585   1.07  
Lukasz Kubot           1360    381   405   1.06  
Ivo Karlovic           1941    299   318   1.06

(In all of these tables, “Games” is the number of service games for that player in the dataset, minimum 1,000 service games. “ExpW” is the expected number of occurences as predicted by the model, “ActW” is the actual number of times it happened, and “Ratio” is the ratio of actual occurences to expected occurences.)

While getting to 15-30 this often is a bit of a disadvantage, it’s one that many of these players are able to erase. Isner, for example, not only remains the favorite at 15-30–his average rate of service points won, 72%, implies that he’ll win 75% of games from 15-30–but from this score, he wins 11% more often than he should.

To varying extents, that’s true of every player on the list. Joao Sousa doesn’t entirely make up for the frequency with which he ends up at 15-30, but he does win 4% more often from 15-30 than he should. Rafael Nadal, Tomas Berdych, and Gael Monfils all win between 6% and 8% more often from 15-30 than the theoretical model suggests that they would. In Nadal’s case, it’s almost certainly related to his skill in the ad court, particularly in saving break points.

At the other extreme, we have players we might term “strong starters” who avoid 15-30 more often than we’d expect. Again, it’s a bit of a mixed bag:

Player                 Games  ExpW  ActW  Ratio  
Dustin Brown           1013    249   216   0.87  
Victor Hanescu         1181    308   274   0.89  
Milos Raonic           3050    514   462   0.90  
Dudi Sela              1066    297   270   0.91  
Richard Gasquet        2897    641   593   0.93  
Juan Martin del Potro  2259    469   438   0.93  
Ernests Gulbis         2308    534   500   0.94  
Kevin Anderson         2946    610   571   0.94  
Nikolay Davydenko      1488    412   388   0.94  
Nicolas Mahut          1344    314   297   0.94

With some exceptions, many of the players on this list are thought to be weak in the clutch. (The Dutch pair of Robin Haase and Igor Sijsling are 12th and 13th.) This makes sense, as the pressure is typically lowest early in games. A player who wins points more often at, say, 15-0 than at 40-30 isn’t going to get much of a reputation for coming through when it counts.

The same analysis for returners isn’t as interesting. Juan Martin del Potro comes up again as one of the players least likely to get to 15-30, and Isner–to my surprise–is one of the most likely. There’s not much of a pattern among the best returners: Novak Djokovic gets to 15-30 2% less often than expected; Nadal 1% less often, Andy Murray exactly as often as expected, and David Ferrer 3% more often.

Before moving on, one final note about reaching 15-30. Returners are much less likely to apply enough pressure to reach 15-30 when they are already in a strong position to win the set. At scores such as 0-4, 0-5, and 1-5, the score reaches 15-30 10% less often than usual. At the other extreme, two of the games in which a 15-30 score is most common are 5-6 and 6-5, when the score reaches 15-30 about 8% more often than usual.

The high-leverage next point

As we’ve seen, there’s a huge difference between winning and losing a 15-30 point. In the 290,000 matches I analyzed for this post, neither the server or returner has an advantage at 15-30. However, some players do perform better than others.

Measured by their success rate serving at 15-30 relative to their typical rate of service points won, here is the top 11, a list unsurprisingly dotted with lefties:

Player             Games  ExpW  ActW  Ratio  
Donald Young       1298    204   229   1.12  
Robin Haase        2134    322   347   1.08  
Steve Johnson      1194    181   195   1.08  
Benoit Paire       1848    313   336   1.08  
Fernando Verdasco  2571    395   423   1.07  
Thomaz Bellucci    1906    300   321   1.07  
John Isner         3166    421   449   1.07  
Xavier Malisse     1125    175   186   1.06  
Vasek Pospisil     1541    243   258   1.06  
Rafael Nadal       3209    470   497   1.06  
Bernard Tomic      2124    328   347   1.06

There’s Isner again, making up for reaching 15-30 more often than he should.

And here are the players who win 15-30 points less often than other service points:

Player                  Games  ExpW  ActW  Ratio  
Carlos Berlocq          1867    303   273   0.90  
Albert Montanes         1183    191   173   0.91  
Kevin Anderson          2946    377   342   0.91  
Guillermo Garcia-Lopez  2356    397   370   0.93  
Roberto Bautista-Agut   1716    264   247   0.93  
Juan Monaco             2326    360   338   0.94  
Matthew Ebden           1088    186   176   0.94  
Grigor Dimitrov         2647    360   341   0.95  
Richard Gasquet         2897    380   360   0.95  
Andy Murray             3416    473   449   0.95

When we turn to return performance at 15-30, the extremes are less interesting. However, returning at this crucial score is something that is at least weakly correlated with overall success: Eight of the current top ten (all but Roger Federer and Milos Raonic) win more 15-30 points than expected. Djokovic wins 4% more than expected, while Nadal and Tomas Berdych win 3% more.

Again, breaking down 15-30 performance by situation is instructive. When the server has a substantial advantage in the set–at scores such as 5-1, 4-0, 3-2, and 3-0–he is less likely to win the 15-30 point. But when the server is trailing by a large margin–0-3, 1-4, 0-4, etc.–he is more likely to win the 15-30 point. This is a bit of evidence, though peripheral, of the difficulty of closing out a set–a subject for another day.

Winning the game from 15-30

For the server, getting to 15-30 isn’t a good idea. But compared to our theoretical model, it isn’t quite as bad as it seems. From 15-30, the server wins 2% more often than the model predicts. While it’s not a large effect, it is a persistent one.

Here are the players who play better than usual from 15-30, winning games much more often than the model predicts they would:

Player             Games  ExpW  ActW  Ratio  
Nikolay Davydenko  1488    194   228   1.17  
Steve Johnson      1194    166   190   1.14  
Donald Young       1298    163   185   1.13  
John Isner         3166    423   470   1.11  
Nicolas Mahut      1344    172   188   1.09  
Benoit Paire       1848    266   288   1.08  
Lukas Lacko        1162    164   177   1.08  
Rafael Nadal       3209    450   484   1.08  
Martin Klizan      1534    201   216   1.08  
Feliciano Lopez    2598    341   367   1.07  
Tomas Berdych      3947    556   597   1.07

Naturally, this list has much in common with that of the players who excel on the 15-30 point itself, including many lefties. The big surprise is Nikolay Davydenko, a player who many regarded as weak in the clutch, and who showed up on one of the first lists among players with questionable reputations in pressure situations. Yet Davydenko–at least at the end of his career–was very effective at times like these.

Another note on Nadal: He is the only player on this list who is also near the top among men who overperform from 15-30 on return. Rafa exceeds expectations in that category by 7%, as well, better than any other player in the last few years.

And finally, here are the players who underperform from 15-30 on serve:

Player               Games  ExpW  ActW  Ratio  
Dustin Brown         1013    122   111   0.91  
Tommy Robredo        2140    289   270   0.93  
Alexandr Dolgopolov  2379    306   288   0.94  
Federico Delbonis    1110    157   148   0.94  
Juan Monaco          2326    304   289   0.95  
Simone Bolelli       1015    132   126   0.96  
Paul-Henri Mathieu   1083    155   148   0.96  
Gilles Muller        1332    179   172   0.96  
Carlos Berlocq       1867    256   246   0.96  
Grigor Dimitrov      2647    333   320   0.96  
Richard Gasquet      2897    352   339   0.96

Tentative conclusions

This is one subject on which the conventional wisdom and statistical analysis agree, at least to a certain extent. 15-30 is a very important point, though in context, it’s no more important than some of the points that follow.

These numbers show that some players are better than others at certain stages within each game. In some cases, the strengths balance out with other weaknesses; in others, the stats may expose pressure situations where a player falters.

While many of the extremes I’ve listed here are significant, it’s important to keep them in context. For the average player, games reach 15-30 about one-quarter of the time, so performing 10% better or worse in these situations affects only one in forty games.

Over the course of a career, it adds up, but we’re rarely going to be able to spot these trends during a single match, or even within a tournament. While outperforming expectations on 15-30 points (or any other small subset) is helpful, it’s rarely something the best players rely on. If you play as well as Djokovic does, you don’t need to play even better in clutch situations. Simply meeting expectations is enough.

The 5 Biggest Comebacks of the 2013 ATP Season

Everybody loves a big comeback, but some of the best come-from-behind wins on the ATP tour this year were such unheralded matchups that they’ve already fallen out of the spotlight.  While everyone else ranks Nadal–Djokovic matches in their year-end lists, let’s look at the five matches in which the winner had to climb out of the biggest hole.

To do this, I ranked every match this season by Comeback Factor (CF), a stat that identifies the lowest ebb in the match for the eventual winner.  If a player breaks serve to open the match and sails to victory, his chance of winning never falls below 50%.  But if he goes down a set and a break, his odds fall much lower.  If the latter player comes back to win, his CF is much higher.

1. Indian Wells Masters R64: Gilles Simon d. Paolo Lorenzi 6-3 3-6 7-5 (win probability graph)

Lorenzi went up a double break in the final set by winning the first four games on the trot.  Simon held twice to force the Italian to serve for it at 5-2.  Lorenzi went up 40-15 in that service game, earning two match points, before losing four points in a row and dropping serve.  At 5-4, Simon broke him to 15, then broke again to love to seal the final set, 7-5.

At 5-2 40-15 in the 3rd set, Lorenzi’s chance of winning was about 99.8%, the highest recorded in a match this year by a player who didn’t end up winning.

2. Queen’s Club R64: Ivan Dodig d. James Ward 6-7(8) 7-6(2) 7-6(2) (win probability graph)

Dodig fought back from nearly the same hole that Simon found himself in, but did so in the second set instead of the third.  Ward won the first set in a tight tiebreak, then earned an early break in the second.  He held on until he served at 5-3, when he reached 40-15.  Dodig won the next four points to erase the break, improving his probability of winning from 0.5% to 21.1%.

Amazingly, the scenario repeated itself in the third set after Dodig won the second in a tiebreak.  Ward went up a break and served for the match again at 5-4, but failed to generate another match point.  The Croatian won a pair of points from 30-30 in that game, then sealed the match in yet another tiebreak.

Dodig wasn’t so lucky a couple of months later, when he nearly upset Juan Martin del Potro in Montreal.  In this year’s 7th-biggest comeback, Delpo came back from a double-break hole in the third set to deny Dodig a place in the third round.

3. Madrid Masters R64: Mikhail Youzhny d. Fabio Fognini 7-6(4) 2-6 7-6(5) (win probability graph)

Fognini never had the double break that led to such disaster for Lorenzi and Ward, but he did have something neither of those men did: a triple match point.  At 3-3 in the deciding set, Fognini broke the Russian then consolidated, leading to a chance to serve for the match at 5-4.  After winning his first three points for a 40-0 advantage, his win probability climbed as high as 99.1%.

It wouldn’t go any higher.  Youzhny won 12 of the next 13 points, breaking the Italian, holding his own serve to love, then earning two match points of his own on the Fognini serve before Fabio gathered himself sufficiently to force a tiebreak.  Fognini kept up his streakiness to the end, claiming a minibreak to open the tiebreak, dropping five points in a row, and fighting back to 5-5 before finally losing the match.

4. Roland Garros R32: Tommy Robredo d. Gael Monfils 2-6 6-7(5) 6-2 7-6(3) 6-2 (win probability graph)

Monfils won the first two sets, which you would think put Robredo at enough of a disadvantage.  But the Spaniard’s lowest ebb didn’t come until the fourth set.  He lost serve in the seventh game, and after fighting off a match point at 3-5, he needed to break serve just to stay alive.

The Frenchman went up 40-15, earning two more match points and a win probability of 98.9%.  Robredo won four straight points to get back on serve, easily held, and even challenged Monfils’s own serve (to 0-30) before landing in a tiebreak.  He won that breaker and, compared to the fourth set, won the fifth with ease.

5. Australian Open QF: David Ferrer d. Nicolas Almagro 4-6 4-6 7-5 7-6(4) 6-2 (win probability graph)

After Robredo beat Monfils, he faced Almagro in the 4th round and Ferrer in the quarters.  Conicidentally, those are the two men who, at the Australian Open, gave 2013 its fifth-biggest comeback.

As in Robredo did in his comeback, Ferrer dropped the first two sets.  Unlike his countryman, he found himself in the most danger in the third set.  Almagro broke in the seventh game of the third set and reached 5-4, an opportunity to serve for the match.  But here, history (or something) got in the way. Almagro reached his highest chance of winning, 98.7%, at 15-0, before Ferrer fought his way to 15-40, Almagro got back to deuce, but Ferrer won the game.

Almagro earned more chances to serve for the match, but his odds of winning would never again be so high.  After breaking in yet another seventh game, Nico served for it at 5-4 and again at 6-5.  At 6-5, he reached 15-0 and a win probability of 97.4%, but from that point on, it was all Ferrer.

Raonic, del Potro, and the Importance of One Point

In last night’s Coupe Rogers match between Milos Raonic and Juan Martin del Potro, one point stands out from the rest.

Raonic won the first set, then Delpo broke early in the second.  With del Potro serving at 4-3, Raonic earned a break point with a winner at the net.  Replays clearly show that he touched the net.  Had the chair umpire seen it in real time, Delpo would have been awarded the point.

The Argentine never recovered, losing the next nine points and the match.

The net touch, and the point Milos didn’t deserve, was clearly a turning point in the match.  But how important was it, really?

If we assume that the two men were equal and that both players win 75% of service points (not true in Delpo’s case yesterday, but reasonable for two big servers on hard courts), here is a summary of Raonic’s probability of winning at various stages of the match:

  • After winning the first set: 75.0%
  • With Delpo serving 4-3, 00-00: 52.4%
  • With Delpo serving 4-3, 40-40: 53.9%
  • After winning the “touch” point: 58.9%
  • If Delpo had won that point: 51.8%
  • After winning the “touch” game: 75.0%
  • After holding serve for 5-4: 76.3%

The controversial point was, clearly, very important.  The difference between winning it and losing it was 7%, a magnitude that doesn’t happen very often in a tennis match, especially outside of tiebreaks.

But the real story here is the next point.  Remember that under normal circumstances, del Potro is a huge server and Raonic does not have a strong return of serve.  (I say “normal circumstances” because somehow, Raonic won 50% of return points in this match.)

If a server is winning 75% of points on his own racquet, his probability of winning a game from break point down is still 67.5%.  There’s a 25% chance he’ll lose the game on the next point, of course, but a 75% chance he’ll get back to deuce, where his serve gives him a 90% chance of winning the game.

The touch point increased Raonic’s chances of winning from 53.9% to 58.9%.  The next point upped his odds from 58.9% to 75.0%.  Which one do you think was more important?

Another way of looking at this to consider what would’ve happened had there been no video replay, and no chance of del Potro spotting the touch and arguing with the umpire about it.  Normal Delpo would’ve stepped back to the line and hit a service winner.  Five minutes later he would’ve held serve again and the two men would’ve played a third set.

It’s easy to look back at this match and conclude that the net touch was the difference in the match.  But no: It was the reaction to the touch–the controversy itself–that had a much greater impact.