Stats
In the early 1990s, my teams (Earth Atomizer and Big Brother) recorded every pass of the season, entering them in a notebook using a shorthand notation during games, and some friends and I would compile them afterwards. Among other things, we found that forehands were thrown away about 50% more frequently than backhands, about 60% of hucks were complete (except for a certain anti-stat hothead who went 4 for 16), and 1.5% of passes were dropped. We did use that first piece of knowledge (coupled with “scouting” observations) about forehands to decide to force forehand most of the time. But what else did we gain for all that time spent?
I have come around to believe that for the time being, the concrete value of tracking individual statistics to predict or to evaluate is doomed by two things, context and sample size. We tried to make one adjustment for context, namely, separating out “tough games” from “chump games”. But then that fed into the second issue, sample size, since we now had fewer games to draw from. And was the line between “tough” and “chump” in the right place? Some of the games were tough because of bad playing conditions, others because we just played badly or exceptionally well in a game that would normally be a blowout, and still others because we had a skeleton crew. Oh, and some teams played zone and forced the handlers to pile up twice as many throws as usual (without, I hope you realize, playing twice as well). But we counted them all equally.
(I should also add that “opportunities” is something that must be accounted for when trying to analyze. But it’s not always as simple as dividing by the number of touches. In the seminal basketball analytics book “Basketball on Paper”, Dean Oliver (now in charge of stats at ESPN) highlighted that player efficiency decreases with increased usage as the players who bear the brunt of the offensive load have to make plays that are closer to the margin. Furthermore, these players will also draw the toughest defenders.)
But we could still tell who was good at completing passes, right? Well, as I like to say at my job where I analyze my company’s engineering performance, it depends. They recorded individual stats on last year’s NexGen tour. I was very excited to get this dataset, because every game was against a quality opponent, almost everyone played almost every game, and each game was a showcase and not just one of many in a long weekend. As it turns out, this dataset too suffers from some confounders such as the first half of the tour beings spent figuring out how to play together and what roles to settle into, but it was still the purest dataset I know of. For the complete tour, turnover percentage* of the players ranged from 3.4% to 12.4%. But for the most part, the guys at the higher end of the turnover range also threw a higher percentage of their passes for goals, while the low-turnover guys didn’t throw as many goals. Here’s the graph for all of them, split out by how often they touched the disc per point:
*They didn’t separate out drops from throwaways so we’ll have to use this instead of incompletion rate.
Note also that the high touch players were in the lower left corner. I can think of two explanations for this besides them being conservative handlers. One, an in-bounds pull almost always results in an uncontested completed pass. Two, passes in general in that half of the field are typically easier to complete because the defense has to respect the threat of the long pass, and I think that handlers have a higher percentage of the touches there than they do closer to the endzone, where pass frequency is more evenly distributed. At the other end of the graph, deeps are going to be catching more of their passes near the endzone, resulting in relatively more opportunities for goal throws but also with each completion a little more difficult because of the reduced space. The risk/benefit of a few extra yards changes near the goal line as well. I wrote in “Ultimate Techniques and Tactics”, a book co-authored with Eric Zaslow and published by Human Kinetics and still available through your favorite Internet reseller, that being in the endzone instead of just on the goal line increases your chance of scoring as much as being 10 yards closer elsewhere on the field.
I once set up a simulation of an offense where the players were equally talented (i.e., had the same incompletion rate per yard of throw) but had different roles in the offense and different throw choices. The first thing I noticed is that a particular player sometimes had MVP-level tournaments and sometimes had tournaments where he would have been benched. The more important point, though, was that the players’ stat lines resembled those of real teams such as NexGen, with some players racking up the goals and turnovers while others had lots of touches but few fantasy league stats. This leads me to conclude that much of the difference between the stat lines of any two players is not a difference in effectiveness but simply a matter of taste. (Note that there are still some players who stand out, either good or bad, but you generally don’t need a calculator to know that.) Two equally-efficient players can have drastically different stat lines due not to any difference in skill or on-field decision-making but to the difference in their roles.
So what do these detailed individual stats (at this stage in our history, where we have only the stats of our own team against a wide range of opponents in vastly different environments) bring to the table? Accountability and self-awareness. Lord Kelvin wrote, “If you cannot measure it, you cannot improve it.” Simply being aware of your actual completion percentage on hucks should force you to contemplate whether you are making good choices. I remember going over each of my turnovers in a weekend (with the aid of the stat pad) and being shocked to learn how many of them were simply poor risk/reward decisions, and I was able to eliminate some of those.
Lest you think I’ve given up on stats, I haven’t. But I think the payoff for now would come on analyzing team decisions. The first priority would be to get realistic baselines for performance. I routinely see people write that five turnovers in a game is typical or that drops never happen or that hucks are completed 75% of the time. While there are certainly examples of these happening, I would guess that they aren’t the typical performance. The other area I would like to quantify the value of particular scenarios. For instance, how much harder is it for a team to score off a deep, high pull vs a low pull vs a brick, and how consistent are good pullers at achieving good pulls? Could someone who is an otherwise bad defender still be a good D player simply by virtue of his pulls? On the offensive side, exactly how costly is it to rest one of your top players? How deadly is it to turn it over in your own half of the field? Might the Huck-‘n’-Hope offensive style actually be a reasonable strategy due to the long field left after a turnover? We might have opinions about those now, but until we measure these, we don’t know.
