How would you assess the quality of play?

This is a difficult task, and would be even if there were a lot of stats available. You would expect that in a growing, still-immature sport that the level of play would increase more or less continuously.

Another confounder is that we’re not sure if we’re talking about individuals or teams, and are we talking about the very best, or the not quite best, or the average? But I'm not going to go into that today.

Zaz talked about this in a different way in his Sept 9 post, “Big Time Sports” and why ultimate is still not ready for prime time.

The following are all indicators, not conclusive proof

• Ability of one player to dominate a game
  
• The degree of specialization
  
• The ability of old and young players to play
  
• The number of players under 5’8”
  
• The number of errors (turnovers)
  
• Percentage of turnovers that are unforced
  
• The ability to score upwind
  
• The ability to move the length of the field
  
• The average margin of victory
  
• The number of ultimate players
  
• The extent of coaches
  
• The number of youth programs
  
• The amount of written instructional material
  
• The sophistication of playbooks and strategies
  

At the cost of belaboring, I’ll explain why each of these is important. I should also caution that I wouldn’t use any of these to compare two individuals or two teams, since there is a lot of noise in the measurement and plenty of counter-examples, but when you have sample size on your side, you ought to be able to see a difference.
Ability of one player to dominate a game. Anthropologist Stephen Jay Gould wrote about this in an article about why there are no more .400 hitters in baseball. When there are more good players in the game, it’s harder for any one of them to rise above. Or look at Colorado’s Beau. He scored 9 goals in the finals of College Nationals. I barely remember his presence when we played them at Club Nationals.
The degree of specialization. The NFL used to have two-way players, and high schools still have a lot of them. Now, there are pass-rushing linebackers and pass-defending linebackers and nickel backs and dime backs and long snappers. Ultimate has more O and D lines now, and players who do only one thing well.
The ability of old and young players to play. If an 18 year old with a year of experience can be a factor at Nationals, no matter how athletic he may be, that says something about the quality of play in the sport. Same deal with a 40 year old, no matter how much he knows about the game. But countering this are a few factors. Players are starting earlier and getting instruction earlier, so 18 might be the new 22. On the other end, I don’t think it ever occurred to most 35 year olds that they should still be playing the game instead of having moved on to life. And for both ends, specialization allows them to contribute what they can without having their weaknesses exploited too much.
The number of players under 5’8”. Short players can compensate for lack of height with quickness, but real athletes are big and quick and would force the little guy out of the game.
The number of errors (turnovers) and the % that are unforced. But an increase in offensive effectiveness could be countered by an increase in defensive effectiveness, leaving the efficiency unchanged. Additionally, a change in the way the rules are enforced (stricter calling of travels, more hacking on the mark) could change the offense/defense balance without being a comment on the skill level. (And let’s not forget the random effects of wind.) You could instead count unforced errors. “Drops” would probably be a good easy-to-track stat. But many “unforced errors” are just good defense, and many “blocks” are just bad throws, so I don’t think you could easily track this. Perhaps “average stall count on turnovers” would be a surrogate. Anyway, having said all this, fewer turnovers generally implies a higher level of play.
The ability to score upwind and the ability to move the length of the field. This seems like an obvious one, but there might actually be limited stats on this. I’m embarrassed for the Open division remembering the DoG/Condors pool play game of 1998 and the Condors/Jam final of 2001 because there were so few upwinders while other divisions had them playing at the same time or on the same field. If “huck it and play zone” is your strategy, that’s a sign of a low level of play (or a 30 mph wind).
The average margin of victory. This could be a measure of parity as much as it is a measure of quality, but more blowouts means that the overall level of play isn’t as high. Little League teams might win one game 18-0 and lose the next 22-2.
The number of ultimate players. More players should equal more good players and more good teams and better top-level teams. But we’d also need to figure out where those players are playing. If we assume no change to the skill of the population but we have twice as many players drawn from that population, we’d have twice as many good players. Are they being spread out to the same number of good teams, or are there twice as many good teams now?
The extent of coaches. Also, the average age of the team captains.
The number of youth programs. This is probably more of a leading indicator, by 5 or 10 years, as it will take some time for these players to develop.
The amount of written instructional material. When everyone has to reinvent the wheel each time, you can’t progress.
The sophistication of playbooks and strategies. If all you’re doing is hucking it and playing zone if it’s not complete, chances are that the level of play isn’t that high.