rookies and cream wrote:Ok, it's been a long time since I reviewed CLT and sampling, so I'll defer to you on those topics. However, as a neuropsychologist (yes, background in stats but less than GTWMA), I interpret standard scores from tests everyday. Although I guess I could be wrong, I do not think you can treat z-scores the same from standard and non-standard distributions, which is essentially what you are doing in the method described. While you are not making interpretations using standard normal tables (e.g., z of 1 = 84th percentile), you are making inferences in regards to player X's standing in a particular category. I do not see how you can make the same assumptions when data in certain categories are not spread evenly.
I think we are saying the same thing, really. A z-score is just a method of standardizing data based on mean and standard deviation. Whether or not you can treat them the same depends on what you intend to do with them. You cannot use z-scores from nonnormal distributions to draw inferences based upon the normal table. But, here we are not drawing inferences across, but within the categories. The assumption we are making is simply that within each category, a player's relative contribution is measured by how many sd above or below the mean they are. And, there's no problem with that. Whether the data are spread evenly or not, we'll be dividing through by the s.d., and that adjusts the valuation within that category for the spread in the data for that category.
I'm trying to think of reasons why the skewness or kurtosis would bias the approach as you add up across categories, but don't see that--but you could be right.
"I don't want to play golf. When I hit a ball, I want someone else to chase it."