I love stats, and always have. When I was 5, I would quiz my mom about stats from the backs of baseball cards. I kept track of Little League stats, wrestling records, golf scores, everything. And in ultimate, I have a record of every tournament since 1992 (and the game scores for those first few years, too). Whenever I have a chance, I've taken individual stats from videos, and was part of a team that recorded every pass back in 1991.
But sometimes I wonder what it's all for. There are a few problems with stats that might just overwhelm their usefulness.
1. Small sample sizes.
2. Other things being equal, ....
3. Field position.
4. False normalization.
The first one is a bane for all sports. You'll hear "so and so is 2 for 18 against this pitcher." This has almost no value in predicting the outcome of their next encounter. Even if you have 100 passes, there is still some chance that a 90% passer will outperform a 95% passer. A lot of studies have shown that the "hot hand" is mostly just a figment of the beholder's imagination.
The problem with the second one is that they're usually not. 3 for 3 is better than 2 for 3, if the passes are the same length, but if the first passer completes dumps that put the disc on a trapped line while the second passer attempts upwind hucks, the second one is more useful. Over time, some of these things wash out, but there will still be outstanding differences resulting from usage patterns. The ideal solution is to compartmentalize the stats as much as possible, so that you separate hucks from dumps, O points from D points, zone from man, strong opponents from weak opponents, but then you get back into problem #1.
Field position (and wind) is just a subset. 40% scoring efficiency might be great if you're starting from your own goal line going upwind, but horrible when starting in the red zone.
False normalization results from the zero-sum aspect of the game. Baseball fielding has a similar problem, in that no matter how bad your defense, you still produce 27 outs in a game. You'll get 15 goals if you win no matter how bad your opponent or how bad you play. Sure, you'll have turnovers, just as you have errors or "defensive efficiency" (percentage of batted balls turned into outs) in baseball, but it makes it impossible to compare players on different teams. Having good teammates will actually hurt your raw stats in either ultimate or baseball (for fielding; batting stats will improve) because you'll have fewer opportunities for yourself.
What to do about this? I'm not sure. It's a bit of a problem, since we need to have a large sample from all levels in order to come up with a good model of how the game works, but no one wants to put in all the effort if they won't be able to do any actionable analysis immediately. But if nobody does anything, then all we're left with is conventional wisdom about what the smart move is in a given situation.
I'd like to see more situational analysis, measuring tradeoffs. For instance, if you have a 20% chance at pulling it OB, is it worth the risk of trying to land it within 10 yards of the sideline? How effective are set plays? What's the difference between having the disc on the line vs in the middle of the field? &c.