Tuesday, March 2, 2010

Luge = Skill; Baseball = Luck?

From guest contributor Sven Wilson:

I had the question of whether the very small differences that separate winning scores from non-winning scores in events like luge are due to randomness or skill? My hypothesis going in was randomness.

However, I got data for the men’s luge on all four runs in the Vancouver games. Using a simple variance-decomposition model, 75% of the variation in run time across the four runs is due to differences between riders and only 25% is due to variation within riders. What that statistical jargon means, essentially, is the 75% of the total variation in run times is due to skill (or possibly equipment), and 25% due to random noise. I was shocked by this. Fortunately, we happened to be talking about just this type of analysis in one of my classes, so I was able to justify spending time on this for legitimate pedagogical purposes.

How well do you think this analysis would hold across other sports? I have argued for some time that in baseball, most of the observed variability is just randomness. In MLB, really good hitters have a .300 percentage and not very good hitters have a .200 percentage. What this means is that for a given at-bat, whether or not a hitter gets on base is mostly determined by the random component, not the skill component. (Note, this is comparing major leaguers to each other, not to ordinary people—in which case the skill component would be relatively more important).

Sven E. Wilson, Associate Professor of Public Policy, Brigham Young University


Corry Cropper said...

Does that model explain how the same players are .300+ hitters year after year, while many others are consistently around .250?

Jeremy said...

@Corry: No. Because BA is calculated over hundreds of trials, error (e) pretty much averages out. Also, in BA there are always (at least) two variables at play: the batter and the pitcher. An IRT analysis would consider the quality of the pitchers each batter faced, and likely produce a more accurate measure of a batter's ability than straight BA.

It's important to note that the 25% variance *within* each lugers' performance is probably an over-estimate because each *run* is also a source of e. Looking at the inter-run correlations, the second run seems to be the outlier (.63 < r < .72, while the other runs had .95 < r < .97).

This is supported by the correlations of the rank-orders for each run. All of them are r > .88. So, even though times varied greatly, the finishing order didn't, pointing to external variables (weather, time of day, etc.) rather than intra-individual differences.

Sven Wilson said...

yea, run 2 is the outlier. Interstingly, though, the mean time for run 2 was the second fastest--but the standard deviation was 1.08 (compared to .77-.84 for the other runs). Something happened in run 2 to drive up the variance in times.

As a result, when I add in fixed effects for individual runs to the simple ANOVA model, the intra-class correlation rises from .738 to only .770, as I would expect because the mean times across runs are pretty similar. It is that higher variation in run 2 that is mucking things up.

In any case, Jeremy, the arguments you make suggest that even higher than 75% (perhaps much higher) of the variation is due to skill or some other intra-rider factor, which is still shocking to me.

Now a question, even if you controlled for pitcher quality, defense quality, game situation, stadium, whether, etc., etc., wouldn't there still be relatively little variation between hitters in MLB? And given that the outcome variable (getting a hit) is dichotomous, any measure of variance you use is going to indicate that skill effects are dominated by the random noise. Am I wrong?

Robert J. Hudson said...

No wonder Lou Gehrig considered himself the "luckiest man on the face of the earth"! Career .340 hitter--he must have never batted against a decent pitcher (because Carl Hubbell's career 2.98 ERA is also lucky). How lucky! This explains everything!!!

Just kidding, Jeremy (and Sven). I appreciate the academic rigor present here--I do; I just get sick of sabermetrics folk trying to reduce all talent to statistical quantification.

Corry Cropper said...

But it does get to the heart of what sport is. You would all probably agree that rolling dice is not a sport. Imagine an event where athletes run through hoops, bike up a mountain, throw a ball, then, to determine the winner of the contest, roll some dice. Is that a sport? It's aerobic, etc. but the outcome is purely random. This is the question Sven is asking about baseball. In one game a batter will make only 4 or 5 plate appearances. The odds of a .300 hitter or a .260 hitter getting on base may be, in fact random, and the result of that specific game may indeed be random. Maybe the difference between batters has more to do with their position in the lineup than their skill... is that possible?

Perry King said...

Good point, Corry.

Line position is one thing, but how about temperament or physical skill? How about the dimensions and design of ballparks, bats and cleats? Or time of day or baseball experience?

There's a lot of random stuff in baseball, but there is some skill within the chaos. Somehow batters are still hitting balls, some more often than others, based on how they study trends and train for situations—e.g bases loaded, Bottom of the 9th, two out.