Friday, November 11, 2005

How would you assess the quality of play?

This is a difficult task, and would be even if there were a lot of stats available. You would expect that in a growing, still-immature sport that the level of play would increase more or less continuously.

Another confounder is that we’re not sure if we’re talking about individuals or teams, and are we talking about the very best, or the not quite best, or the average? But I'm not going to go into that today.

Zaz talked about this in a different way in his Sept 9 post, “Big Time Sports” and why ultimate is still not ready for prime time.

The following are all indicators, not conclusive proof

  • Ability of one player to dominate a game
  • The degree of specialization
  • The ability of old and young players to play
  • The number of players under 5’8”
  • The number of errors (turnovers)
  • Percentage of turnovers that are unforced
  • The ability to score upwind
  • The ability to move the length of the field
  • The average margin of victory
  • The number of ultimate players
  • The extent of coaches
  • The number of youth programs
  • The amount of written instructional material
  • The sophistication of playbooks and strategies

At the cost of belaboring, I’ll explain why each of these is important. I should also caution that I wouldn’t use any of these to compare two individuals or two teams, since there is a lot of noise in the measurement and plenty of counter-examples, but when you have sample size on your side, you ought to be able to see a difference.
Ability of one player to dominate a game. Anthropologist Stephen Jay Gould wrote about this in an article about why there are no more .400 hitters in baseball. When there are more good players in the game, it’s harder for any one of them to rise above. Or look at Colorado’s Beau. He scored 9 goals in the finals of College Nationals. I barely remember his presence when we played them at Club Nationals.
The degree of specialization. The NFL used to have two-way players, and high schools still have a lot of them. Now, there are pass-rushing linebackers and pass-defending linebackers and nickel backs and dime backs and long snappers. Ultimate has more O and D lines now, and players who do only one thing well.
The ability of old and young players to play. If an 18 year old with a year of experience can be a factor at Nationals, no matter how athletic he may be, that says something about the quality of play in the sport. Same deal with a 40 year old, no matter how much he knows about the game. But countering this are a few factors. Players are starting earlier and getting instruction earlier, so 18 might be the new 22. On the other end, I don’t think it ever occurred to most 35 year olds that they should still be playing the game instead of having moved on to life. And for both ends, specialization allows them to contribute what they can without having their weaknesses exploited too much.
The number of players under 5’8”. Short players can compensate for lack of height with quickness, but real athletes are big and quick and would force the little guy out of the game.
The number of errors (turnovers) and the % that are unforced. But an increase in offensive effectiveness could be countered by an increase in defensive effectiveness, leaving the efficiency unchanged. Additionally, a change in the way the rules are enforced (stricter calling of travels, more hacking on the mark) could change the offense/defense balance without being a comment on the skill level. (And let’s not forget the random effects of wind.) You could instead count unforced errors. “Drops” would probably be a good easy-to-track stat. But many “unforced errors” are just good defense, and many “blocks” are just bad throws, so I don’t think you could easily track this. Perhaps “average stall count on turnovers” would be a surrogate. Anyway, having said all this, fewer turnovers generally implies a higher level of play.
The ability to score upwind and the ability to move the length of the field. This seems like an obvious one, but there might actually be limited stats on this. I’m embarrassed for the Open division remembering the DoG/Condors pool play game of 1998 and the Condors/Jam final of 2001 because there were so few upwinders while other divisions had them playing at the same time or on the same field. If “huck it and play zone” is your strategy, that’s a sign of a low level of play (or a 30 mph wind).
The average margin of victory. This could be a measure of parity as much as it is a measure of quality, but more blowouts means that the overall level of play isn’t as high. Little League teams might win one game 18-0 and lose the next 22-2.
The number of ultimate players. More players should equal more good players and more good teams and better top-level teams. But we’d also need to figure out where those players are playing. If we assume no change to the skill of the population but we have twice as many players drawn from that population, we’d have twice as many good players. Are they being spread out to the same number of good teams, or are there twice as many good teams now?
The extent of coaches. Also, the average age of the team captains.
The number of youth programs. This is probably more of a leading indicator, by 5 or 10 years, as it will take some time for these players to develop.
The amount of written instructional material. When everyone has to reinvent the wheel each time, you can’t progress.
The sophistication of playbooks and strategies. If all you’re doing is hucking it and playing zone if it’s not complete, chances are that the level of play isn’t that high.


Sam TH said...

I'm surprised that Beau wasn't a factor in your game against Bravo. He was certainly a big factor in the Bravo-Jam game, although obviously not the way he was in college.

gcooke said...

Some quick comments:

The Degree of Specialization- This is an interesting point. Football is certainly a model of this. Other sports that have "arrived", like soccer and basketball, have, in my opinion, less specialization than football. I would attribute some of this to a) more continuity of action, b) fewer substitutions, and c) more changes of possession.

# of errors- It is interesting to watch socccer with an Ultimate perspective toward turnovers.

Amount of written material- It will interesting to see if a few different systems attain prominence in Ultimate. Currently, you might go to NC, and have folks yell "air" instead of "up". Or you might play on Short Fat Guys and when they say "right" off the pull, that means where the play is going...not where the stack should be.
The basic fundamentals driving these subtle variations are pretty much the same from team to team.

I think most players are trying to learn what the wheel is...and are not aware that it is being reinvented.

j-co said...

s. j. gould was a biologist and paleontologist, jim. important figure in contemporary evolutionary theory. and he was a big baseball fan.

also, i think high school ultimate is where it's at in terms of pushing the sport forward. hmmm... another post idea for me.

Justin R said...

A little close to home but I would add:

* The number of above average yet still mediocre athletes who's only connection to the high level game is through the internet.

Guilty as accused.

Anonymous said...

Young players are fairly prevalent in a lot of sports we might consider advanced:
Olympic sports (gymnastics, etc)

parinella said...

The list is primarily to compare a sport to itself over time, or to compare different levels of the sport. But I think the youth examples cited (other than gymnastics, where flexibility is the most important attribute) lend credence to the list. If a 14 year old girl can compete with the world's best at tennis or golf or any sport that requires power, the sport has a lot of room to improve. Compare men's tennis or golf to women's, for instance.

re soccer, while there are a million kids playing the game, they're not doing it at a high level. There is Freddy Adu, but he's just one player, and aren't almost all World Cup players in their 20s?

And the Olympic sports have young players, but that's because there is no professional outlet for them, so once they leave college they stop playing competitively.

Dan said...

Hi -- Lots of the big football (soccer) stars debut as full professionals in there teams. I think you're argument about youth is wrong.
A small list:
Cristiano Ronaldo 16 (full Portugal Intl by 18)
Ronaldo 16 (went to WC for Brazil at 17)
Wayne Rooney 16 (full England internatioal at 17)
David Beckham 17 (full england international at 20)
Michael Owen 16 (full England international by 18)
Pele 16 (full brazil international at 17 when he scored 6 goals at WC)

parinella said...

I don't really give a rat's ass about soccer, but I know three of these guys. Their prominence at as teenagers doesn't say much about the argument; if there are a lot of teenagers who can play at a World Cup level but who are not future inner-circle Hall of Famers, then THAT would be a good rebuttal to my argument. In top level club ultimate these days, there seem to be a bunch of guys who are currently playing in college.

Edward Lee said...

And the Olympic sports have young players, but that's because there is no professional outlet for them, so once they leave college they stop playing competitively.

Lots of Olympic athletes (mostly in Communist countries) are essentially pro athletes, subsidized by their governments. But the sports in which kids do well are sports like gymnastics, figure skating and swimming/diving.

I think what you want to do is to limit your scope to sports that require high levels of each of three elements: fitness, "skill" and tactical/strategic thought.

sometallskinnykid said...

How about a lack of dominant players in their prime (meaning late 20s/early 30s)?

Furious is the closest to this and they have won 3 out of the last 4, pretty impressive. I believe "the fab 5" are all 28 or older.

But what other teams have that where the main parts are in their prime (assuming your prime hits in late 20s/early 30s)? It seemed the core of the Condors during their 2 titles were in this age range, but now they have a younger team (or so I here).

Sockeye definitely had more players in this age range then this year. But still their main guys of the last 2 years are all 26 or younger. I feel the same way about Jam too.

I mean, what are all these guys doing? Or did Dog just stomp them out early?

Dan said...

> if there are a lot of teenagers > who can play at a World Cup
> level but who are not future
> inner-circle Hall of Famers,
> then THAT would be a good
> rebuttal to my argument
You've set the bar pretty high -- high level pro debuts more relevant than International debuts (which mean you're one of the best 20-odd players in a country) to compare to ultimate. You're asking not who would make a nationals bound club roster, but who would be nominated as one the 20 best ultimate players in the country.

But you're still wrong. This is from the 19 man outfield roster for England's international against Argentina last week:
7 (36%) debuted for England as teens
8 (79%) debuted before their 23rd birthday (I.e. college age)
Average age of debut for roster: 20.7

parinella said...

Well, OK, Dan, you've shown that there are a bunch of soccer players who were able to play as teens. But these guys are the top 20 in the country, so many of them were probably good enough to be the top x when they were teenagers. Are there a lot of teenagers in the top English league, or is it that those who are there are just so good that they stand out? The NBA only occasionally gets a teenager, but it's a Lebron James or Amare Stoudamire who is ready to play there.

If all of a sudden there were 2 or 3 teenagers per NBA team, the most likely explanation would be either that the level of play dropped significantly or that the level of teenagers' play increased significantly. With ultimate, this latter explanation may be possible, as we are just now starting to see a lot of young players who have been playing for 8 or 10 years, or perhaps some great athletes who are getting good coaching and picking the game up more quickly. But we might also just be seeing great athletes who don't know the game that well but are able to play because the level of play has dropped.

Dan said...

I wonder how many teenagers were at nationals this year... Does anyone know? Would also be interested in what kind of roles they played. I'd also love to know the average age, but i doubt that's easily available either.

Basketball is so much more "mature" as a sport. So while your argument makes sense for them (i.e. explosion of teens) I disagree it makes sense for us.

I would assume the athletic ability of the average junior ultimate player is much better now than 5 years ago. And yes, he's getting better coaching.

But this doesn't mean quality at the top is worse, or we've got a freakishly good group of kids. It's just that ultimate is reaching down to that younger age. I think it's our destiny to always have a couple of teenagers on each team at the top -- just as it is today in soccer, and probably basketball soon. Hell, even the big bruisers in the NHL probably have at least one 19 year old per team.

Let's put it this way - if you had played juniors ultimate of today's quality starting when you were 13, don't you think you would have made it to nationals by 18? You probably would have been better at your peak, too.

Again, Soccer is the perfect example -- the most rigorously scouted and competitve game on the planet. Yet every big English (premier league) soccer club has a couple of teenagers playing every year (and in soccer remember you only get 3 subs a game total -- so you don't have the option of easing people in that ultimate provides). This kids have the speed and skills already. Sure they're not as tactically smart, but they'll get there. You're right that the best of the best players are mid-late 20s. But that's probably how it is ultimate too.

If we end up with teenage MVPS, or rosters full of them, then I'll agree somethings wrong.