Bouchard, Halep, and First-Time Quarterfinalists

Two of the final eight women in Melbourne, Eugenie Bouchard and Simona Halep, are playing in their first Grand Slam quarterfinals. Let’s take a look at how other women have done in their first appearances this late in a Slam.

In the Open era, 267 different women have reached the final eight of a Slam. At the time of their debut quarterfinal, their average age was roughly 21 years and four months. Their average WTA ranking was 42, not considering those who predated the ranking system or those who reached their first quarterfinal as an unranked player.

Of the 267, 197 (73.8%) progressed no further in their breakthrough slam. 52 (26.4%) won one more match, losing in the semifinals; 12 (6.1%) reached the final but lost; and the remaining six players won the title when the reached their first Open-era quarter.

However small 6 of 297 sounds, such an outcome is actually even rarer. Three of those six first-time quarterfinalists don’t really count–they reached their first QF in 1968, the first year of the Open era. Billie Jean King, winner of the Australian Open that year, isn’t that great a comp for Bouchard or Halep. The only other players to win a Grand Slam in their first quarterfinal appearance are Chris O’Neil (1978 Australian), Barbara Jordan (1980 Australian), and Serena Williams (1999 US Open).

While we can’t count on Bouchard or Halep winning the tournament this week, their appearances in Slam quarterfinals at relative young ages bodes well. The earlier a player reaches her first major QF, the more QFs she is likely to reach over the course of her career.  In fact, of the 22 women who have reached more than 10 Slam quarterfinals since 1984, only one of them–Jana Novotna–failed to reach her first one in her teens. She didn’t make it until the ripe old age of 20 years and 8 months.

Bouchard has just snuck in before her 20th birthday, which she’ll celebrate next month. Her most age-appropriate comp is Victoria Azarenka, who reached her first major quarterfinal–at the 2009 French Open–just a few weeks younger than Genie is now. Less than five years later, Vika will play her 12th Slam QF.

Less optimistic comparisons for Bouchard are Yanina Wickmayer and Anna Chakvetadze, both of whom reached their first major quarterfinal in the last two months of their teens. Chakvetadze made two more final eights; Wickmayer is still looking for her second.

If history is any guide, Halep’s prospects are bleaker. At 22 years and four months, she is much older than any of the players who have reached double-digit Slam quarterfinals except for Li Na, who is playing in her 10th QF this week. Li didn’t play in the final eight of a Grand Slam until she was 24 years old.

The 61 players who reached their first Slam QF at an older age than Halep did not, on average, achieve much more. They’ve totaled 81 additional QFs–well below two per person.

Of course, the age profile of the WTA is changing, so a 22-year-old debutante isn’t nearly the oddity it was a decade or two ago. It’s no coincidence that Halep’s most optimistic comp is Li, an active player. That’s the most positive outlook for the Romanian, anyway. To rack up an impressive career record, she’ll have to follow Li’s lead and overcome a late start.

—

The ATP final eight also features a newbie, Grigor Dimitrov. The changing age profile of the ATP is even more drastic, so age-based analysis is less meaningful. But we can take a quick look at the precedents for the Bulgarian’s first Slam quarterfinal.

There have been 329 ATP Slam quarterfinalists in the Open era, and first-timers stand a better chance in the men’s game. 32.5% of debut Slam quarterfinalists have advanced to the semis, and 13 of them (4.0%) went on to win the tournament. Then again, none of them had to beat Rafael Nadal in the quarters.

While Dimitrov is older than Halep–and as noted, 22-year-olds didn’t used to be considered so young on the ATP tour–there are some positive examples for Grigor to follow.

Michael Stich reached his first Slam QF at almost exactly the same age as Dimitrov is now, and he not only reached the semis at that event (the 1991 French Open), but qualified for the final eight in nine more majors. Jo Wilfried Tsonga, David Ferrer, and Nikolay Davydenko all reached their first Slam QF later than Dimitrov, and each has played in the final eight at least ten times.

On average, those optimistic comps are outweighed by all the guys who made it to one or two Slam QFs later in their career. The 153 players who reached their first final eight later than Dimitrov’s current age have returned to a total of 362 additional quarterfinals–good for one or two more appearances per player.

Despite all the hype, Dimitrov’s performance this year isn’t a drastic breakthrough. It’s only a single step in the right direction–especially considering that he reached this milestone by beating the #73 player in the world. He could be the next Tsonga, or he could be the next Robby Ginepri.

The Limited Value of Head-to-Head Records

Italian translation at settesei.it

Yesterday at the Australian Open, Ana Ivanovic defeated Serena Williams, despite having failed to take a set in four previous meetings. Later in the day, Tomas Berdych beat Kevin Anderson for the tenth straight time.

Commentators and bettors love head-to-head records. You’ll often hear people say, “tennis is a game of matchups,” which, I suppose, is hardly disprovable.

But how much do head-to-head records really mean?  If Player A has a better record than Player B but Player B has won the majority of their career meetings, who do you pick? To what extent does head-to-head record trump everything (or anything) else?

It’s important to remember that, most of the time, head-to-head records don’t clash with any other measurement of relative skill. On the ATP tour, head-to-head record agrees with relative ranking 69% of the time–that is, the player who is leading the H2H is also the one with the better record. When a pair of players have faced each other five or more times, H2H agrees with relative ranking 75% of the time.

Usually, then, the head-to-head record is right. It’s less clear whether it adds anything to our understanding. Sure, Rafael Nadal owns Stanislas Wawrinka, but would we expect anything much different from the matchup of a dominant number one and a steady-but-unspectacular number eight?

H2H against the rankings

If head-to-head records have much value, we’d expect them–at least for some subset of matches–to outperform the ATP rankings. That’s a pretty low bar–the official rankings are riddled with limitations that keep them from being very predictive.

To see if H2Hs met that standard, I looked at ATP tour-level matches since 1996. For each match, I recorded whether the winner was ranked higher than his opponent and what his head-to-head record was against that opponent. (I didn’t consider matches outside of the ATP tour in calculating head-to-heads.)

Thus, for each head-to-head record (for instance, five wins in eight career meetings), we can determine how many the H2H-favored player won, how many the higher-ranked player won, and so on.

For instance, I found 1,040 matches in which one of the players had beaten his opponent in exactly four of their previous five meetings.  65.0% of those matches went the way of the player favored by the head-to-head record, while 68.8% went to the higher-ranked player. (54.5% of the matches fell in both categories.)

Things get more interesting in the 258 matches in which the two metrics did not agree.  When the player with the 4-1 record was lower in the rankings, he won only 109 (42.2%) of those matchups. In other words, at least in this group of matches, you’d be better off going with ATP rankings than with head-to-head results.

Broader view, similar conclusions

For almost every head-to-head record, the findings are the same. There were 26 head-to-head records–everything from 1-0 to 7-3–for which we have at least 100 matches worth of results, and in 20 of them, the player with the higher ranking did better than the player with the better head-to-head.  In 19 of the 26 groups, when the ranking disagreed with the head-to-head, ranking was a more accurate predictor of the outcome.

If we tally the results for head-to-heads with at least five meetings, we get an overall picture of how these two approaches perform. 68.5% of the time, the player with the higher ranking wins, while 66.0% of the time, the match goes to the man who leads in the head-to-head. When the head-to-head and the relative ranking don’t match, ranking proves to be the better indicator 56.5% of the time.

The most extreme head-to-heads–that is, undefeated pairings such as 7-0, 8-0, and so on, are the only groups in which H2H consistently tells us more than ATP ranking does.  80% of the time, these matches go to the higher-ranked player, while 81.9% of the time, the undefeated man prevails. In the 78 matches for which H2H and ranking don’t agree, H2H is a better predictor exactly two-thirds of the time.

Explanations against intuition

When you weigh a head-to-head record more heavily than a pair of ATP rankings, you’re relying on a very small sample instead of a very big one. Yes, that small sample may be much better targeted, but it is also very small.

Not only is the sample small, often it is not as applicable as you might think. When Roger Federer defeated Lleyton Hewitt in the fourth round of the 2004 Australian Open, he had beaten the Aussie only twice in nine career meetings. Yet at that point in their careers, the 22-year-old, #2-ranked Fed was clearly in the ascendancy while Hewitt was having difficulty keeping up. Even though most of their prior meetings had been on the same surface and Hewitt had won the three most recent encounters, that small subset of Roger’s performances did not account for his steady improvement.

The most recent Fed-Hewitt meeting is another good illustration. Entering the Brisbane final, Roger had won 15 of their previous 16 matches, but while Hewitt has maintained a middle-of-the-pack level for the last several years, Federer has declined. Despite having played 26 times in their careers before the Brisbane final, none of those contests had come in the last two years.

Whether it’s surface, recency, injury, weather conditions, or any one of dozens of other factors, head-to-heads are riddled with external factors. That’s the problem with any small sample size–the noise is much more likely to overwhelm the signal. If noise can win out in the extensive Fed-Hewitt head-to-head, most one-on-one records don’t stand a chance.

Any set of rankings, whether the ATP’s points system or my somewhat more sophisticated (and more predictive) jrank algorithm, takes into account every match both players have been involved in for a fairly long stretch of time. In most cases, having all that perspective on both players’ current levels is much more valuable than a noise-ridden handful of matches. If head-to-heads can’t beat ATP rankings, they would look even worse against a better algorithm.

Some players surely do have an edge on particular opponents or types of opponents, whether it’s Andy Murray with lefties or David Ferrer with Nicolas Almagro. But most of the time, those edges are reflected in the rankings–even if the rankings don’t explicitly set out to incorporate such things.

Next time Kevin Anderson draws Berdych, he should take heart. His odds of beating the Czech next time aren’t that much different from any other man ranked around #20 against someone in the bottom half of the top ten. Even accounting for the slight effect I’ve observed in undefeated head-to-heads, a lopsided one-on-one record isn’t fate.

Should WTA Players Approach the Net More?

Italian translation at settesei.it

21st-century women’s tennis is a baseline game. Some players are better able to identify opportunities to approach the net than others, and some can handle themselves quite well when they get there. But if a fan from a few decades ago were dropped off at the 2014 Australian Open, she would be shocked by the rarity of net points and the clumsiness of many players when they move forward.

Since almost all television commentators were excellent players in a more net-centric era, a frequent refrain during almost any broadcast is that players should rush the net more often. “Frequent” might be understating it–in a fit of pique, I was driven to say this:

Regardless of repetition, it’s worth further investigation. It’s certainly true that a skilled netwoman could win more points by moving forward. But when pros don’t emphasize that part of their game and they gain little match experience approaching the net, do they have the skills necessary to take advantage of such an opportunity?

Enter some numbers

At this point, you might be tempted to look at the oft-collected “Net Points” stat. Resist the urge. In a baseline-oriented match, net points can have little to do with net approaches. Attempting to return a drop shot is considered a net point. Putting away a weak service return is considered a net point. In many WTA matches, more than half of “net points” do not involve an approach. The player was induced to come to the net for some reason.

Making matters worse, that non-approach segment of net points has little to do with net approaches. Given a weak, floating return, any competent player should be able to whack it for a swinging volley winner. At the other end of the spectrum, chasing down a drop shot relies on a different set of skills than picking a moment to hit an approach shot and then confidently placing a volley or two.

Fortunately, the Match Charting Project gives us some more detailed, approach-specific data.

Twenty matches in the charting database are from the first month of the 2014 WTA season, most of them from the first week in Melbourne. This data differentiates between “net approaches” and “net points.” In one of the more aggressive performances in the database, Angelique Kerber, in her loss to Tsvetana Pironkova in Sydney, won 15 of 19 net points. Of her ten net approaches, she won all ten.

(For any match report in the charting database–here’s the Kerber-Pironkova match–click one of the two “Net Points” links to see those stats. There is a different table for each player.)

Kerber’s ten net approaches is tied for the most of any of the WTA matches that have been charted this year. Last night, Garbine Muguruza also tallied ten net approaches, though she did so in a longer match.

In these twenty matches, only 27 of 40 players made even one traditional net approach. Including those who made zero, the average is just over three net approaches per match. The 27 who approached the net at least once averaged 4.7 per match.

Clearly, a lot of opportunities for offense are going unclaimed.

How they’re doing

Of the 126 net approaches we’ve tracked, the approaching player has won 84–exactly two-thirds. While that isn’t an overwhelming endorsement–many approach shots are hit in response to a weak groundstroke that already puts the opponent at a disadvantage–it certainly doesn’t count as evidence against the practice.

In half of all net approaches, the netrusher either hits an outright winner at the net or induces a forced error with a net shot.  Only 12% of the time does the opponent hit a passing shot winner. In another 5% of these points, the opponent induces a forced error with a passing shot. In 12% of net approach points, the player who moved forward hits an unforced error at the net.

Of the 27 players in the database who approached the net at least once, only six failed to win half of those points (three of whom only came forward once), and three more won exactly half of their net approach points.

The women in this sample who seize the most opportunities to rush the net have been particularly successful, as well. Seven of the eight who moved forward the most won more than half of their approach points.  This allows us to tentatively conclude that all the other players–the ones who picked only a few spots to approach the net during their matches–could have seized more opportunities. There may be a limit in the modern game to how much netrushing is wise, but the observed maximum of ten points per match doesn’t seem to be it.

Inevitable unknowns

Whether we look at Kerber and her 10/10 net-approach performance in Sydney or Sloane Stephens and her 1/1 tally yesterday against Elina Svitolina, it’s impossible to know the results of the next approach shot–or the next five.  We can compare single-match results and see that it’s possible for a WTA player to have a perfect record on her ten net approaches, but we can’t perform lab experiments in which Sloane plays Svitolina again and comes forward ten times instead of one.

For all the success that players enjoy when they do move forward, there are plenty of reasons not to. As I said at the outset, today’s players don’t practice net skills nearly as much as baseline skills, and they certainly don’t get much in-match practice. If someone isn’t comfortable approaching the net at a certain time, is it really a good idea for her to do so?

In the abstract, both intuition and statistical analysis supports the position that WTA players could move forward more. When they do approach the net, they are often successful, putting away volley winners and rarely getting passed. But I suspect this implies a long-term strategy more than the sort of thing a coach should emphasize during a changeover.

When commentators suggest that a player should move forward, what I think they really mean is this: “If this player were more comfortable with her transition game, this would be a great opportunity to take advantage of that.” Or: “Players should work harder on their approach shots on the practice court so that they’re ready for opportunities like this one.” Or simply: “Martina would have won that point ten shots ago.”

There seems to be opportunity waiting for more, well, opportunistic young players. But it isn’t one that can be generated simply by a sudden coaching change or a harangue from John McEnroe. Only when a player emerges with the baseline game to contend with the best pros and a transition/net game that exceeds most of those on the tour today will we find out just how much opportunity today’s players have wasted.

Novak Djokovic and a First-Serve Key to the Match

Landing lots of first serves is a good thing, right? Actually, how much it matters–even whether it matters–depends on who you’re talking about.

When I criticized IBM’s Keys To the Match after last year’s US Open, I identified first-serve percentage as one of three “generic keys” (along with first-serve points won and second-serve points won) that, when combined, did a better job of predicting the outcome of matches than IBM’s allegedly more sophisticated markers.  First-serve percentage is the weakest of the three generic keys–after all, the other two count points won which, short of counting sets, is as relevant as you can get.

First-serve percentage is a particularly appealing key because it is entirely dependent on one player. While a server may change his strategy based on the returning skills of his opponent, the returner has nothing to do with whether or not first serves go in the box.  Unlike the other two generic targets and the vast majority of IBM’s keys, a first-serve percentage goal is truly actionable: it is entirely within one player’s control to achieve.

In general, first-serve percentage correlates very strongly with winning percentage.  On the ATP tour from 2010 to 2013, when a player made exactly half of his first serves, he won 42.8% of the time. At 60% first serves in, he won 47.0% of the time. At 70%, the winning percentage is 57.4%.

This graph shows the rates at which players win matches when their first-serve percentages are between 50% and 72%:

1svAs the first-serve percentage increases on the horizontal axis, winning percentage steadily rises as well.  With real-world tennis data, you’ll rarely see a relationship much clearer than this one.

Different players, different keys

When we use the same approach to look at specific players, the message starts to get muddled.  Here’s the same data for Novak Djokovic, 2009-13:

nd1sv

While we shouldn’t read too much into any particular jag in this graph, it’s clear that the overall trend is very different from the first graph. Calculate the correlation coefficient, and we find that Djokovic’s winning percentage has a negative relationship with his first-serve percentage. All else equal, he’s slightly more likely to win matches when he makes fewer first serves.

Djokovic isn’t alone in displaying this sort of negative relationship, either. The three tour regulars with even more extreme profiles over the last five years are Marin Cilic, Gilles Simon, and the always-unique John Isner.

Isner regularly posts first-serve percentages well above those of other players, including 39 career matches in which he topped 75%. That sort of number would be a near guarantee of victory for most players–for instance, Andy Murray is 32-3 in matches when he hits at least 70% of first serves in–but Isner has only won 62% of his 75%+ performances.  He is nearly as good (57%) when landing 65% or fewer of his first serves.

Djokovic, Isner, and this handful of others reveals a topic on which the tennis conventional wisdom can tie itself in knots. You need to make your first serve, but your first serve also needs to be a weapon, so you can’t take too much off of it.

The specific implied relationship–that every player has a “sweet spot” between giving up too much power and missing too many first serves–doesn’t show up in the numbers. But it does seem that different players face different risks.  The typical pro could stand to make more first serves. But a few guys find that their results improve when they make fewer–presumably because they’re take more risks in an attempt to hit better ones.

Demonstrating the key

Of the players who made the cut for this study–at least 10 matches each at 10 different first-serve-percentage levels in the last five years–9 of 21 display relationships between first-serve percentage and winning percentage at least as positive as Isner’s is negative.  The most traditional player in that regard is Philipp Kohlschreiber. His graph looks a bit like a horse:

pk1sv

More than any other player, Kohli’s results have a fairly clear-cut inflection point. While it’s obscured a bit by the noisy dip at 64%, the German wins far more matches when he reaches 65% than when he doesn’t.

Kohlschreiber is joined by a group almost as motley as the one that sits at the other extreme. The other players with the strongest positive relationships between first serve percentage and winning percentage are Richard Gasquet, Murray, Roger Federer, Jeremy Chardy, and Juan Martin del Potro.

These player-specific findings tell us that in some matchups, we’ll have to be a little more subtle in what we look for from each guy. When Murray plays Djokovic, we should keep an eye on the first-serve percentages of both competitors–the one to see that he’s making enough, and the other to check that he isn’t making too many.

Better at Best-of-Five

Italian translation at settesei.it

The best-of-five-sets format used in Grand Slam men’s singles favors the mentally and physically strong. It also gives better players the edge, as it reduces the number of fluky results.

However, simple best-of-five records aren’t always our most useful guide. A player who consistently goes deep in slams faces difficult opponents far more frequently than he would at ATP 250 or 500 events.  We would expect that many players would have worse records in best-of-five matches not because of any tendency, but because of consistently tough draws.

If we accounted for all that–opponent quality and the structural bias toward favorites in best-of-fives–who would come out strongest? Which players outperform expectations the most in Grand Slams?

Let’s start with a few names you might not expect, before we narrow the search down to players with the longest resumes:

  • Of players with 100 career tour-level matches, the man who has outperformed the most at Slams is Bernard Tomic. In 35 matches at majors, he has won 20 despite his rankings suggesting he would win only 11–82% better than expected. Outside of slams, he has precisely played to his ranking.  Modest as Bernie’s track record is, no other active player comes close to this gap.
  • Bump the threshold to 200 career matches, and your man is … Denis Istomin? His 21-21 record at Slams doesn’t seem so impressive until you consider that he has never been seeded. His rankings would imply he should have won only 16 matches.
  • The parade of underdogs continues when we up the standard to 300 career matches. Victor Hanescu has outperformed best-of-five expectations by a solid 20%, going 28-32 while his lowly rankings would suggest he should have won only 23 matches.

Let’s move on to the big dogs. I meant to limit this study to active players, but when you go far enough back to cover the careers of guys like Radek Stepanek and Tommy Haas, you end up getting a lot of notable former stars. And here, Marat Safin stands out.

Safin’s career mark in slams was 95-41, excellent by any standard. His winning percentage of about 70% was about 14% higher than the combination of his rankings and his opponents would have predicted. While he exceeded expectations in Slams, he underwhelmed in other events.  He won almost 10% fewer best-of-three matches than would have been expected over the course of his career.

No current top-ten player displays as big a gap between best-of-five and best-of-three performance than Safin did, but Jo Wilfried Tsonga comes the closest. In this table, I’ve shown each player’s career record at majors, how that compares to the number of wins they should have expected, then the same pair of numbers for non-slams. (I’ve excluded all Davis Cup matches.) Finally, the “ExpRat” column shows how much better each guy played at majors than at non-majors–the ratio of how much more the player exceeded expectations at Slams than elsewhere.

Player                 Bo5 W%  Bo5 Exp  Bo3 W%  Bo3 Exp  ExpRat  
Jo Wilfried Tsonga      76.6%     1.17   66.6%     0.98    1.19  
Tomas Berdych           69.3%     1.06   63.1%     0.95    1.12  
Stanislas Wawrinka      67.0%     1.17   58.6%     1.08    1.08  
Novak Djokovic          85.3%     1.06   79.2%     0.99    1.07  
Rafael Nadal            88.3%     1.07   82.1%     1.01    1.06  
Andy Murray             79.6%     1.05   73.7%     1.03    1.02  
Roger Federer           84.4%     1.01   79.2%     1.00    1.01  
David Ferrer            70.6%     0.95   65.6%     0.96    0.99  
Richard Gasquet         64.0%     1.04   63.5%     1.09    0.95  
Juan Martin Del Potro   72.9%     1.13   72.5%     1.24    0.91

It’s inevitable that the Big Four make up the middle of the pack. When you are as good as they have been for as long as they have been, you can only exceed expectations so much. Most impressive of the group is Rafael Nadal, who has been better at majors than non-majors despite vastly preferring clay courts. Many of the journeyman players who do the worst by this metric are clay specialists–guys like Filippo Volandri and Potito Starace who are virtually guaranteed first-round exits at three slams each year.

While Juan Martin del Potro‘s appearance at the bottom of this list seems particularly timely after last night’s upset, it may be no more than a statistical fluke stemming from his long absence and long comeback in 2010 and 2011.  His ranking lagged behind his skill level for a long time, which explains why he has managed to exceed expectations both at Slams (+13%) and at non-majors (+24%). The loss to Roberto Bautista Agut won’t help his numbers, but it may take several more slams before we can be more confident about Delpo’s best-of-five tendencies.

When Tsonga and Roger Federer clashed in last year’s Melbourne quarterfinals, Federer escaped in five sets. If, as expected, they face off this year as well, these numbers represent one reason why Fed might not be so lucky again.

—

For further reading, check out Colin Davy’s similar study.

The Changing Depth of the WTA

During yesterday’s broadcast of the Australian Open match between Alison Riske and Yanina Wickmayer, commentator Elise Burgin discussed whether the depth of the WTA has increased over the years. She felt strongly that it has, and she had a very useful illustration on screen, as 55th-ranked Riske was putting on an impressive display of shotmaking en route to a 6-1 6-1 victory.

From a quantitative perspective, “depth” can be hard to pin down. If lower-ranked players are holding their own against the top five, or ten, or thirty, it could mean that the field is very deep, or it could mean that we’re in an era without all-time greats.  As Burgin pointed out, the WTA might not currently have a top five to match those of some recent eras, but there’s little doubt that today’s top two could line up with just about any of the last few decades.

It would be very difficult to settle whether today’s top ranks are good, bad, or otherwise in historical terms, so for now, let’s assume they are average.  We’ll return to that in a bit.

Let’s start by looking at how the WTA top 32 has fared against everybody else. This encompasses about 900 matches per season. The trend isn’t overwhelming, but it does seem that the top 32 is not quite as dominant as it was in some previous periods:

depth32

The 2012 and 2013 winning percentages of 73.4% and 74.7% represented the lowest two-year span since 1984 (where my ranking database begins).  Aside from the outlier years of 2004 and 2007, the top 32 has won fewer than 77% of its matches against the pack for more than a decade.  In the 1980s and 1990s, the top 32 was consistently above that number.

Of course, drawing the line at the top 32 is arbitrary. Most of us would think of the 19th- or 26th-ranked player as part of the pack, not as a defining player of this generation.  Let’s see how the graph looks if we draw the line at the top 10:

depth10

Looking at the top 10 against everyone else doesn’t differentiate the current era quite as much as the top 32 does, but it continues to show that the pack is quite competitive in historical terms.

Since 1984, the top 10 has won almost exactly 80% of matches against everyone else, and for the last two years, the WTA has matched that number.  However, in the very recent past, from 2009 to 2011, the pack posted the three best single-season records against the top 10, peaking in 2010, when the top 10 won only 74% of matches against others.

As I noted at the outset, comparing “the top” with “the pack” in a series of years implies that one or the other is a constant. The top–especially a small group such as the top 10–almost certainly isn’t. In 2010, that great season for the pack, Serena Williams played only 29 matches, compared to 62 in 2009 and 82 in 2013. Add another 30 or 50 Serena matches to the sample and maybe the pack wouldn’t have looked so good.

While the pack is less affected by single injuries, it probably isn’t a constant either. After all, the claim that launched this post is that the pack has improved.  Thus, we can’t entirely trust these numbers as a rating of the top based on their record against the pack, or as a rating of the pack based on their record against the top.

However, we can see broad trends and supplement them with some qualitative judgments.  If you believe that today’s top ten is a particularly weak one, the fact that the pack is winning only 20% of their matches against that group isn’t exactly an endorsement. If you think the top of the game is particularly strong, that 20% looks much better, supporting Burgin’s position that the pack is better than ever.

An alternative theory that may explain this intuition about the pack is based on injuries. WTA injury numbers (based on retirements and withdrawals, anyway) are at an all-time high, and advances in sports medicine are getting players back on court quicker than ever. Thus, there is always a pool of players whose talent level is not represented by their ranking, either because they are injury prone and never reach that ranking, or because they’ve recently missed time and seen their ranking fall during that period.

Of course, there have always been players in the field returning from injury, but at any given time, there are probably more today than there were twenty years ago. And that means more unseeded, lower-ranked competitors with the capability of beating a top player. They usually don’t–as in the cases of Andrea Petkovic, Venus Williams, and Vera Zvonareva this week–but if you’re looking at a draw hunting for dark horses and interesting early-round matchups, those are the sorts of names that deliver.

Given all the moving parts in this sort of analysis, it’s tough to draw conclusions. If a couple of players suddenly emerge as dominant players and complement Serena and Vika at the top of the game, we could see these numbers swing in favor of the top. If Serena suddenly retires, they’ll probably swing in favor of the pack. For now, the best I can offer is that the pack–whether defined as those outside the top 10, top 32, or any number in between–is probably a bit better than the WTA’s historical average.

The Geriatric Australian Open

You’ve probably heard about the steady aging of professional tennis.  In both the men’s and women’s games, fewer teenagers than ever are winning important matches, and more and more thirty-somethings are remaining at the top of the game.

My favorite illustration: 25 years ago, the oldest man in the Australian Open draw was Johan Kriek, about two months short of his 31st birthday when the tournament began.  This year, 24 men in the main draw are older.

A total of 33 men in the singles draw have reached their fourth decade, only the third time in tournament history that the number has exceeded 20.  If lucky loser Stephane Robert replaces the injured Gilles Simon, we’ll have 34 thirty-somethings, tied with the all-time record, set in 2012.

Even without Simon’s withdrawal, we already have a record for average age in the men’s draw.  That figure this year is 27 years and 126 days, 80 days more than the previous record, set last year.  (Replacing Simon with Robert would add another 11 days to the average.) The new record also marks the seventh consecutive year that the average age of the men’s singles draw has increased.

While the age of the women’s draw isn’t quite record-setting, the rise of thirty-somethings in the women’s game has been even more rapid.  Only 13 years ago, in 2001, Els Callens was the only woman over the age of 30 in the draw (she was a mere 156 days past her 30th birthday).  This year, there are a record-high 15 players over the age of 30 in the women’s singles draw.

The 2012 Aussie Open field remains the oldest on record, at 24 years and 321 days.  This year’s draw–at 24 years and 292 days–is close enough that, had 16-year-old Ana Konjuh lost her third-round qualifying match to Olga Savchuk, ten years her senior, we would be looking at a new record.

Long term trends and the folly of forecasting

By just about any metric you might devise, the game has gotten steadily older for about 25 years.  As with any trend in the news, this one has led too many commentators–both casual and more academic–to claim that this is a permanent trend, or that “you’ll never see another teenage tennis champ.”

Protip: Never put your money on “never.”

What these arguments often fail to account for is that, for about twenty years after the inception of the pro game in the late 1960s, the sport–both men’s and women’s–consistently got younger.  When the 2012 Wimbledon men’s draw broke that event’s record for average age, the record it was breaking was from 1968.

Sure, there are plenty of possible explanations for the steady age decline of the 1970s and 1980s, just as there are many for the current increase.  And there are probably hard limits at either extreme that prevent the age of the game from swinging too far in either direction.

In any case, we’re not in the middle of an infinite rise in ages any more than we were amid an endless decline in 1985.  Twenty years from now, the 2014 Aussie Open data points could be an meaningless step on this upward path or an important inflection point in another shift in the game.  We’re unlikely to see a teenage Slam champ next year, or the year after that, but is it really possible to make a sensible case that, in six years, today’s 12-year-olds will be helpless against today’s 24-year-olds?

What we can be confident about is what has happened, and even without accounting for the return of Pat Rafter, this year’s Melbourne field represents yet another data point in the aging of elite-level tennis.

—

Detailed stats: Lots of great things are happening with the Match Charting Project. Several people have stepped forward and started contributing to the project already this year, and we’re up to 144 matches in the database.  From Day One in Australia: Bencic vs Date-Krumm, Venus vs Makarova, and Errani vs Goerges.  I hope you’ll join in the fun.

Winners and Losers in the 2014 Australian Open Men’s Draw

Every draw carries with it plenty of luck, but even by Grand Slam standards, this year’s Australian Open men’s singles draw seems a bit lopsided.  The top half makes possible a Rafael Nadal–Roger Federer semifinal, at least if Federer gets past Andy Murray and Nadal beats the likes of Bernard Tomic.

While Novak Djokovic is seeded below Nadal, he gets the benefit of a projected semifinal matchup with David Ferrer.  A more substantial challenge may arise one round earlier, as a possible quarterfinal opponent is Stanislas Wwrinka, who took Djokovic to a fifth set twice in the last four Grand Slams.

As I’ve done in the past, let’s quantify each player’s draw luck.  Using my forecast, combined with a forecast generated by randomizing the bracket, we can see who were the biggest winners and losers in yesterday’s draw ceremony.

The algorithmic approach is most useful in confirming our suspicions about the draw luck of the top players.  Djokovic and Ferrer, the top seeds in the bottom half, definitely came out ahead.  While Djokovic had a respectable 28.0% chance of winning the tournament in the randomized projection, he has a 33.7% chance given the way the draw turned out.  In turns of expected ranking points, the draw gave him a 10.7% boost, from an expectation of 747 points to one of 827 points.  In percentage terms, Ferrer’s expectation jumped even more, from 312 to 368 (18.0%).

Nadal, however, had the worst draw luck of the top ten seeds.  Before the bracket was arranged, he had a 30.7% chance of winning the title, with an expectation of 763 ranking points.  Once the draw was set, his title chances fell to 24.9% and his point expectation dropped to 662.  No one else in the top ten lost more than 7% of their expected ranking points on draw day; Nadal lost 13%.

It doesn’t take an algorithm, though, to identify the draw’s worst losers.  They’re placed where you’ll always find them: right next to the top two seeds.  In the randomized projection, Tomic had a 58% chance of winning his first-round match and a 27% chance of reaching the third round.  In reality, though, he’ll play Nadal first.  His slight chance of earning a place in the second round gives him an expectation of 29 ranking points (10 of which he earns simply by showing up).  In the random projection, his ranking point expectation was 75.

Lukas Lacko, the unlucky man who will play Djokovic in the first round, didn’t suffer quite so much, if only because he didn’t have as high of expectations in the first place.  Before the draw, he could expect 48 ranking points and a 15% chance of reaching the third round.  Now, his projection is a mere 24 ranking points, one of the worst in the entire draw.

The luckiest players are always those who had little chance of progressing far in the draw, but managed to draw someone equally inept.  At the Australian Open, the four luckiest guys have yet to be identified: all are qualifiers.  The luckiest man of all will be the one who is placed in the topmost qualifying spot, opposite Lucas Pouille.  At this stage, my rating system doesn’t think much of the Frenchman, so it is likely that the qualifier will be the heavy favorite entering that match.

In the randomized projection, each qualifier has a 29% chance of winning his first match and a 6% chance of winning his second, for a weighted average of 32 ranking points.  The man who plays Pouille, however, will enter the field with an expectation of 55 ranking points.  Other qualifiers with nearly the same happy outcome will be those who draw Federico Delbonis, Julian Reister, and Jan Hajek in the opening round.

Here are the pre-draw and post-draw expected ranking points of the men’s seeds, along with the percentage of pre-draw points they gained or lost:

Player                 Seed  Pre  Post  Change  
Rafael Nadal           1     763   662  -13.2%  
Novak Djokovic         2     747   827   10.7%  
David Ferrer           3     312   368   18.0%  
Andy Murray            4     473   488    3.1%  
Juan Martin Del Potro  5     421   393   -6.6%  
Roger Federer          6     411   397   -3.4%  
Tomas Berdych          7     264   317   20.2%  
Stanislas Wawrinka     8     290   279   -3.9%  

Player                 Seed  Pre  Post  Change
Richard Gasquet        9     186   186    0.1%  
Jo Wilfried Tsonga     10    151   187   23.8%  
Milos Raonic           11    223   234    5.0%  
Tommy Haas             12    207   222    7.5%  
John Isner             13    176   196   11.2%  
Mikhail Youzhny        14    190   193    1.5%  
Fabio Fognini          15    101    81  -19.3%  
Kei Nishikori          16    172   135  -21.6%  

Player                 Seed  Pre  Post  Change
Tommy Robredo          17     71    61  -13.4%  
Gilles Simon           18    116    95  -18.3%  
Kevin Anderson         19     80   107   33.9%  
Jerzy Janowicz         20     99   154   55.3%  
Philipp Kohlschreiber  21    125   132    6.2%  
Grigor Dimitrov        22    136   122  -10.1%  
Ernests Gulbis         23    125   107  -14.1%  
Andreas Seppi          24     94    49  -47.8%  

Player                 Seed  Pre  Post  Change
Gael Monfils           25    147   101  -31.4%  
Feliciano Lopez        26    100    80  -20.7%  
Benoit Paire           27     94    89   -5.5%  
Vasek Pospisil         28     82    81   -0.9%  
Jeremy Chardy          29    111   126   13.7%  
Dmitry Tursunov        30    101    80  -21.0%  
Fernando Verdasco      31    106   105   -0.8%  
Ivan Dodig             32    104   106    1.8%

Men, Women, and Unforced Errors

Italian translation at settesei.it

If you’ve ever suffered through a debate about the relative merits of men’s and women’s tennis, you’ve probably heard the assertion that women’s tennis is sloppier–“riddled with unforced errors,” perhaps.  Maybe you’ve even made that claim yourself, which is understandable, given how often some version of it crops up, unchallenged, in tennis commentary.

But is it really true?  Do WTA matches feature so many more unforced errors than ATP matches? Unforced errors were counted at most slam matches last year, so we can find out.

Let’s start with the most recent results.  In men’s matches at the 2013 US Open, 33.2% of points ended in an unforced error.  Play may have tightened up just a bit in the final week: In the round of 16 and later, 32.9% of points ended in UFEs.

Women’s matches did, in fact, feature a higher rate of unforced errors. Considering the entire tournament, 39.7% of points ended that way, while in the fourth round and later, the rate dropped to 36.7%.

So yes, there are more unforced errors in the women’s game.  There are similar gaps between ATP and WTA error rates at Wimbledon and the Australian Open, and while the difference on the French Open clay is smaller, it is still present.

Eyeballing errors

However, these aren’t massive differences.  Using the US Open numbers, we can calculate that WTA points ended in UFEs about 20% more often than ATP points.  In the last four rounds of the tournament, when more people are watching closely and drawing conclusions, that difference drops to 11.7%.

Without a scorebook in hand, that gap may well be too small to spot.  In a typical set of, say, 60 points, the average ATP pairing averaged 20 UFEs, against a typical WTA matchup’s  24.  That’s one extra unforced error every other game–if that.  Looking at the four final rounds, the difference drops to 20 UFEs in a men’s match against 22 in a women’s match.  Two extra errors a set.

The divide is real, but it hardly seems substantial enough to represent a major difference in the quality of play or in the viewing experience.

Here are the numbers for the entire field at all four 2013 slams, followed by the rates in the final 16:

Slam             ATP UFE%  WTA UFE%  WTA/ATP  
Australian Open  36.2%        44.4%     1.22  
French Open      33.6%        37.0%     1.10  
Wimbledon        19.1%        24.6%     1.29  
US Open          33.2%        39.7%     1.20

R16 and later:                                           
Slam             ATP UFE%  WTA UFE%  WTA/ATP  
Australian Open  36.4%        41.1%     1.13  
French Open      33.9%        34.9%     1.03  
Wimbledon        20.5%        24.4%     1.19  
US Open          32.9%        36.8%     1.12

Don’t read too much into the contrasts between one slam and another–what’s important here is how the same set of scorers, in the same conditions, are judging men’s and women’s matches.  Wimbledon, especially, is known for its, shall we say, unique approach to counting unforced errors.

Instead, a power gap

The French Open rates are by far the closest of those at the four slams.  This shouldn’t come as a surprise.  On a slower surface, ATPers earn fewer free points than usual on serve, finding themselves more frequently in rallies.  Take away those one- or two-shot rallies that the men’s game is known for, and the UFE disparity starts to shrink.

While we can’t account for all service winners and forced error returns, we can take aces out of the equation.  So far, we’ve only see unforced errors as a percentage of all points.  Take UFEs as a percentage of all non-ace points, and the difference between men’s and women’s error rates decreases.

In other words, now we’re starting to look at what happens when the serve is returnable:

Slam             ATP UFE%  WTA UFE%  WTA/ATP  
Australian Open  39.6%        46.2%     1.17  
French Open      35.6%        38.3%     1.08  
Wimbledon        21.2%        25.9%     1.22  
US Open          36.1%        41.3%     1.14  

R16 and later:                                
Slam             ATP UFE%  WTA UFE%  WTA/ATP  
Australian Open  39.6%        42.8%     1.08  
French Open      35.3%        36.0%     1.02  
Wimbledon        22.7%        25.6%     1.13  
US Open          34.9%        38.3%     1.10

In most of these cases, we’re down to a couple of points per set.  If we were able to sort out service winners and perhaps forced error returns, we would almost surely see even more minor differences.

There’s no doubt that men hit harder serves and are, on average, more likely to win a point without having to hit a second ball.  But if we’re comparing the characteristics of women’s tennis, it doesn’t seem right to give the men credit for not hitting as many unforced errors when some of the already modest difference is due to the dominance of the serve.

Quibbles

This entire analysis depends on the unforced error stat, which I don’t much care for.  It is hugely dependent on the scorer, and there’s no widespread agreement in the sport on what exactly it means.

However, if we want to challenge a widely-held belief about unforced errors, there’s not really any way around using unforced errors, is there?

The best we can do to eliminate scorer’s biases is to compare only within single events.  The same person isn’t counting unforced errors at every US Open match, but each scorer probably works both men’s and women’s matches.  At a given venue, every scorer might even go through the same training program.

Even with that consideration, there is the strong possibility that scorers make adjustments–consciously or unconsciously–depending on the gender of players on court.  If unforced errors are shots that a player should have made but didn’t, a lot hinges on your interpretation of the word “should.”  It may be that some shots would be called unforced errors in a men’s match, but forced errors in a women’s match.  To the extent that’s the case, it’s awfully difficult to compare the genders using a stat that itself differs depending on gender.

On the other hand, scorers are presumably tennis fans, and they’ve heard the same conventional wisdom everyone else has.  If you believe that women hit more unforced errors than men do, perhaps you call borderline women’s shots unforced and borderline men’s shots forced.  In that case, scorers might be unwittingly amplifying the gender difference, not reducing it.

Given the difficulties of collecting data from hundreds of matches on different continents spread across many months, I doubt any non-automated method of counting unforced errors would address all of these issues.  For now, we have to take the official unforced error counts as the best available representation of reality and draw conclusions accordingly.

Whatever the limitations of the data, and whatever the other differences between the genders on a tennis court, unforced error counts are not nearly the distinguishing factor that they’ve been made out to be.

Analytics That Aren’t: Why I’m Not Excited about SAP in Tennis

It’s not analytics, it’s marketing.

The Grand Slams (with IBM) and now the WTA (with SAP) are claiming to deliver powerful analytics to tennis fans.  And it’s certainly true that IBM and SAP collect way more data than the tours would without them.  But what happens to that data?  What analytics do fans actually get?

Based on our experience after several years of IBM working with the Slams and Hawkeye operating at top tournaments, the answers aren’t very promising.  IBM tracks lots of interesting stats, makes some shiny graphs available during matches, and the end result of all this is … Keys to the Match?

Once matches are over and the performance of the Keys to the Match are (blessedly) forgotten, all that data goes into a black hole.

Here’s the message: IBM collects the data. IBM analyzes the data. IBM owns the data. IBM plasters their logo and their “Big Data” slogans all over anything that contains any part of the data. The tournaments and tours are complicit in this: IBM signs a big contract, makes their analytics part of their marketing, and the tournaments and tours consider it a big step forward for tennis analysis.

Sometimes, marketing-driven analytics can be fun.  It gives some fans what they want–counts of forehand winners, or average first-serve speeds. But let’s not fool ourselves. What IBM offers isn’t advancing our knowledge of tennis. In fact, it may be strengthening the same false beliefs that analytical work should be correcting.

SAP: Same Story (So Far)

Early evidence suggests that SAP, in its partnership with the WTA, will follow exactly the same model:

SAP will provide the media with insightful and easily consumable post-match notes which offer point-by-point analysis via a simple point tracker, highlight key events in the match, and compare previous head-to-head and 2013 season performance statistics.

“Easily consumable” is code for “we decide what the narratives are, and we come up with numbers to amplify those narratives.”

Narrative-driven analytics are just as bad–and perhaps more insidious–than marketing-driven analytics, which are simply useless.  The amount of raw data generated in a tennis match is enormous, which is why TV broadcasts give us the same small tidbits of Hawkeye data: distance run during a point, average rally hit point, and so on.  So, under the weight of all those possibilities, why not just find the numbers that support the prevailing narrative? The media will cite those numbers, the fans will feel edified, and SAP will get its name dropped all over the place.

What we’re missing here is context.  Take this SAP-generated stat from a writeup on the WTA site:

The first promising sign for Sharapova against Kanepi was her rally hit point. Sharapova made contact with the ball 76% of the time behind the baseline compared to 89% for her opponent. It doesn’t matter so much what the percentage is – only that it is better than the person standing on the other side of the net.

Is that actually true? I don’t think anyone has ever published any research on whether rally hit point correlates with winning, though it seems sensible enough. In any case, these numbers are crying out for more context.  Is 76% good for Maria? How about keeping her opponent behind the baseline 89% of the time? Is the gap between 76% and 89% particularly large on the WTA? Does Maria’s rally hit point in one match tell us anything about her likely rally hit point in her next match?  After all, the article purports to offer “keys to match” for Maria against her next opponent, Serena Williams.

Here’s another one:

There is a lot to be said for winning the first point of your own service game and that rung true for Sharapova in her quarterfinal. When she won the opening point in 11 of her service games she went on to win nine of those games.

Is there any evidence that winning your first point is more valuable than, say, winning your second point?  Does Sharapova typically have a tough time winning her opening service point?  Is Kanepi a notably difficult returner on the deuce side, or early in games?  “There is a lot to be said” means, roughly, that “we hear this claim a lot, and SAP generated this stat.”

In any type of analytical work, context is everything.  Narrative-driven analytics strip out all context.

The alternative

IBM, SAP, and Hawkeye are tracking a huge amount of tennis data.  For the most part, the raw data is inaccessible to researchers.  The outsiders who are most likely to provide the context that tennis stats so desperately need just don’t have the tools to evaluate these narrative-driven offerings.

Other sporting organizations–notably Major League Baseball–make huge amounts of raw data available.  All this data makes fans more engaged, not less. It’s simply another way for the tours to get fans excited about the game. Statheads–and the lovely people who read their blogs–buy tickets too.

So, SAP, how about it?  Make your branded graphics for TV broadcasts. Provide your easily consumable stats for the media.  But while you’re at it, make your raw data available for independent researchers. That’s something we should all be able to get excited about.