Return to Baseball Leftovers

Reliability of small samples

Moderator: Baseball Moderators

Reliability of small samples

Postby Ender » Fri May 10, 2013 11:25 am ... leid=20516

I just wanted to post this because I've argued against the numbers that had been posted in the past and the guy who did the original study redid it and found how completely flawed his earlier results were.

I'd post the table itself but I don't know how to do that accurately~. The one that always bugged me was that LD% was supposed to be reliable at 150 PA which is just obviously not true, the new study shows LD% is more like 650 to 1000 balls in play just as an example of just how wrong some of the numbers that have been posted have been.

K and BB stabilize fast, everything else is a small sample really far into the season. BABIP takes 2000 balls in play to stabilize as an extreme example. Anyway for stat guys this is a must read imo, especially if you have used his old numbers at some point.
Hall of Fame Hero
Hall of Fame Hero

CafeholicFantasy Expert
Posts: 7705
(Past Year: -28)
Joined: 30 Apr 2004
Home Cafe: Baseball

Re: Reliability of small samples

Postby Skin Blues » Fri May 10, 2013 11:34 am

Tango Tiger has posted the table from that BP article as well, after changing it to r=.5 instead of r=.7 ... half-noise

And the article for the hitting statistics is here.

Yeah, the LD% data was way off the first time. Don't know how that happened as it seems rather obvious that LD rate is extremely unstable. It's a good lesson to not just blindly trust any piece of research on the internet. The new ones seem to have a lot more reliability though, as the methods have been greatly improved and are well established, from what I understand.

Also, the data for pitchers is kind of deceiving as it seems that K% would be reliable after 70 PA, for example, but it's only reliable assuming that he faced a representative sample of hitters. Which is of course not true, especially for starting pitchers who may face the same weak team a couple times early on, accounting for the vast majority of their first 70 PA against. It would be nice to have a good "quality of competition" stat that could be used to roughly adjust a player's output. I'm sure it already exists somewhere.
Skin Blues
General Manager
General Manager

User avatar

Posts: 3082
(Past Year: 82)
Joined: 11 Apr 2010
Home Cafe: Baseball

Return to Baseball Leftovers

Who is online

Users browsing this forum: No registered users and 0 guests