Defensive Metrics thread continued - Fantasy Baseball Cafe 2014 Fantasy Baseball Cafe
100% Deposit Bonus for Cafe Members!

Return to Baseball Leftovers

Defensive Metrics thread continued

Moderator: Baseball Moderators

Re: Defensive Metrics thread continued

Postby bigwords » Tue Jan 06, 2009 5:40 pm

We can agree on that last statement.
bigwords
Major League Manager
Major League Manager


Posts: 1026
Joined: 7 Jun 2006
Home Cafe: Baseball

Re: Defensive Metrics thread continued

Postby GotowarMissAgnes » Tue Jan 06, 2009 5:56 pm

Big Pimpin wrote:

So I pulled the data and played with LF (I only used qualified players). What I found was interesting. There is some phenomena with the 2008 data that is messing with your quick and dirty CV. It was indeed around 9% in 2008, but it was much higher in 2007 (28%) and 2006 (34%). I left 2004-05 out because it's very apparent that something changed dramatically in the measure between 2005 and 2006.

I also pulled the offensive numbers for the same set of players (I got a few more on the offensive side for some reason, probably they were listed primarily as LF, had enough ABs to qualify, but didn't spend enough time in the field to qualify in the fielding data). I looked at the same Q&D CV for BA, OBP, SLG, and OPS, and found that the ZR CV wasquite comparable. BA was around 32%, OBP was around 22% (2006 was like 38% but there were some big outliers), SLG was around 55%, and OPS was in the 33%-35% range (2006: 45%). So in general, I did not find that offensive CV were much larger.

I also had to do some research into ZR because I've never used it. I'm not sure how much I really trust it that much because it doesn't use batted ball data (with the exception of what is classified as "in zone" relative to "out of zone"). [Sidenote: The THT article talking about ZR even go so far as to say that systems like PMR are an improvement.] So say a player has an abnormal percentage of balls hit into the far reaches of "his zone" as opposed to another who has most of the balls hit relatively close to where he's positioned. The former is naturally going to have a worse ZR simply because the data drives it. Now, if you want to convert ZR and OOZ plays into a +/- system (which others have done) then you're on the right track and have something much more meaningful.

There's no doubt there are some variations across players and years, I just don't see any reason that it should be a huge worry.


I think we'll need to compare notes.

I agree that ZR is not the best measure, but unlike some of the later models, it gives a clean number that is comparable to offensive rate stats in terms of success divided by opportunities. We could switch metrics and do this more in terms of a runs driven framework like VORP or something, but the same result would occur.

As you did I only used qualified players. I am also using the Revised Zone Ratings (RZR) measures on Hardball Times. And I do the comparison position by position, because fielding statistics are so position dependent. I think you also want to exclude Manny from those other years or else use a different measure of variance than range. Range is highly sensitive to outliers, and so Manny's bizarre performance, I think, accounts for your finding on the difference between 2008 and other years (Manny does not qualify in the 2008 data). If you exclude Manny or if you use a measure of variance that is not so sensitive to his outlier status and go through any year for LF, RF, and CF, you'll find that fielding variance ranges around half of those offensive metrics.
"I don't want to play golf. When I hit a ball, I want someone else to chase it."
GotowarMissAgnes
Hall of Fame Hero
Hall of Fame Hero

User avatar
Fantasy Expert
Posts: 5516
Joined: 12 Dec 2003
Home Cafe: Baseball
Location: Happy Valley

Re: Defensive Metrics thread continued

Postby Big Pimpin » Tue Jan 06, 2009 6:05 pm

noseeum wrote:
bigwords wrote:Interesting discussion.

Just to add 2 cents.

There seems to be an assumption, particularly in some of the comments I've seen from Noseum, that those who believe in defensive metrics treat offense and defense with equal weight when adding up a player's value. I don't believe this is true. He says that "Defensive skill does not automatically translate into runs saved" and where defense may play a part in runs saved, that it is "a smaller overall component of runs saved, than offensive skill is for runs scored." I'm not sure about the total accuracy of those statements but I will say that the measurement of defense has been quite more dynamic than assumed, and shouldn't be assumed to be weighted on a 1:1 ratio with a player's offensive contributions. Don't think that makes a player's defensive contributions more difficult to quantify as much as it's just a possibly smaller component of a player's overall value. And as Big Pimpin pointed out, it could very well be the case that a player's defensive contributions is a more stable force year-to-year. I haven't done research on that, but it would seem to make a little bit of logical sense.

In any case, I don't think it's any accident that many of the most sabermetric-inclined teams including the A's, Red Sox, Blue Jays, Padres, etc. have placed a great deal of priority on defense in recent years. Nor do I think it's a coincidence that all but one of last year's playoff teams including Tampa Bay Rays (ranking #1), the Cubs (ranking #2), Red Sox (#4), Brewers (#7), Phillies (#10), Angels (14), and Dodgers (15) ranked in the top half of defensive efficiency last season. You might see such events as the beginning of circumstantial evidence that the value of defense is bearing out.


I'm definitely not saying that defense is considered to have a 1:1 relationship to hitting. I'm saying the precise relationship is not known. I'm saying that a sum total number of a player's value in a past season has so many assumptions built into it that it's impossible to trust it.

There are about 7 respected evaluations of defensive value. Anyone who's come up with a sum total statistic which included defense has hopefully chosen one of those 7. But even those 7 will come up with vastly different estimates for a specific player based on the same exact season. One metric will place a player at below average while another will place him above average. This just does not happen on offense.

If we can't even get agreement on how good a player was in a past year defensively, than there is no way we can trust that we have a true sense of what percentage of that player's value can be attributed to defense. And further, there is no way we can even pretend we can predict how many runs he could save us next year.

A metric built upon numerous assumptions is a house of cards. See standard deviation in the financial markets for where that can get you.


I actually do think that hitting and defense are basically 1:1 and should be given equal weight. That being said, I don't think we can completely derive a player's skill level, only a range around how many runs that player will save (or cost) a team in a given year.

As far as the systems go, I think the respected number is closer to four. UZR, PMR, Chone, and Dewan's +/-. I'd love to hear what the are that you will call respected, as well as some examples of players they greatly disagree on. For your example of Ichiro, I pulled the numbers from 2005-2008 (based on RF because he only has one year in CF, and I rounded) from UZR, PMR, and Chone (no Fielding Bible laying around)-
UZR: +2, +12, +8, +6 (avg: 7)
PMR: +22, +10, +35, +9 (avg: 19)
Chone (2009 Projection based on 2005-2008 data): +7

Now, knowing the things we do about Ichiro, we can adjust this. PMR loves him in CF but not as much in RF, so I would probably adjust his 2006-2008 numbers down a bit because he played at least part of the year in CF. So maybe I'm happy with the +7 idea. I'd probably be fine with that, slap a range on it, then add a couple runs because of his arm. So I'd call him a +5 to +15 RF and be pretty happy with that estimate.

I know there are players where different opinions come up, but most of those guys are right around average, so they might be a bit higher or a bit lower. Of the guys I've looked at there haven't been any radically divergent opinions.

As far as your below vs. above average, I think that does happen on offense because there isn't one universally accepted measure (hopefully wOBA will be it). One person might value BA, another OBP, another values SLG, another OPS and another OPS+. Just as bigwords said, everyone has an opinion on what's the right measure is so how can there possibly be agreement. As an example, Mike Cameron was widely regarded as a disappointing offensive player during his time in Seattle. I could dig up the Bavasi quote from when he kicked him to the curb, but it doesn't matter. He had a low average and he struck out a lot. The fact is that he played in a park that greatly reduced his offensive effectiveness, and yet his bat was still worth at least a win on it's own merit, notwithstanding the fact that he was the best defensive CF alive during the time.
Big Pimpin
Mod in Retirement
Mod in Retirement

User avatar
EditorCafeholicFantasy ExpertCafe WriterCafe RankerGraphics ExpertMock(ing) DrafterEagle EyeWeb SupporterMatchup Meltdown ChampionPick 3 Weekly WinnerLucky Ladders Weekly Winner
Posts: 13710
(Past Year: 5)
Joined: 20 Apr 2005
Home Cafe: Baseball
Location: Building a metric. And being ignorable and stupid.

Re: Defensive Metrics thread continued

Postby Big Pimpin » Tue Jan 06, 2009 6:07 pm

noseeum wrote:I guess you did miss the original. I said there is no way that Bradley + RP is better than Dunn. Big Pimpin replied that he believed he proved it to be true. I told him he didn't prove anything. He just proved that using certain systems, Bradley + RP nets you a higher number than Dunn. I said to me that proves the system is flawed, not that Dunn is worse.


I also said several times I don't think I "proved" that Bradley + RP was better than Dunn, simply that they were comparable. And the only thing I guaranteed were that actual results would vary. ;-)
Big Pimpin
Mod in Retirement
Mod in Retirement

User avatar
EditorCafeholicFantasy ExpertCafe WriterCafe RankerGraphics ExpertMock(ing) DrafterEagle EyeWeb SupporterMatchup Meltdown ChampionPick 3 Weekly WinnerLucky Ladders Weekly Winner
Posts: 13710
(Past Year: 5)
Joined: 20 Apr 2005
Home Cafe: Baseball
Location: Building a metric. And being ignorable and stupid.

Re: Defensive Metrics thread continued

Postby noseeum » Tue Jan 06, 2009 6:09 pm

Here's a discussion on all the defensive metrics available:
http://ussmariner.com/2006/01/24/evaluating-defense/
noseeum
Major League Manager
Major League Manager


Posts: 1697
Joined: 1 Mar 2005
Home Cafe: Baseball

Re: Defensive Metrics thread continued

Postby noseeum » Tue Jan 06, 2009 6:22 pm

Big Pimpin wrote:As far as your below vs. above average, I think that does happen on offense because there isn't one universally accepted measure (hopefully wOBA will be it). One person might value BA, another OBP, another values SLG, another OPS and another OPS+. Just as bigwords said, everyone has an opinion on what's the right measure is so how can there possibly be agreement. As an example, Mike Cameron was widely regarded as a disappointing offensive player during his time in Seattle. I could dig up the Bavasi quote from when he kicked him to the curb, but it doesn't matter. He had a low average and he struck out a lot. The fact is that he played in a park that greatly reduced his offensive effectiveness, and yet his bat was still worth at least a win on it's own merit, notwithstanding the fact that he was the best defensive CF alive during the time.


This is just not true. If someone still holds the opinion that BA is a more important stat than OBP, than that person is misinformed. Yes, there are various "opinions" about offense, but most of those opinions are wrong. Sure, baseball writers can think Jim Rice was great, but the numbers say he wasn't.

OBP is the most important offensive stat. It correllates most highly with runs. That is not an opinion. That's why OPS is a simple yet inaccurate stat. It overweights slugging. These are not opinions. They are facts.

Yes, there is research at the margins arguing what's the best sum total for offensive evaluation, OPS+, Win Shares, WOBA, etc. But those are summaries of statistics that are beyond dispute. This player walked up to bat 650 times last year, he walked 120 times, he stole 10 bases, he hit 40 homers, 36 doubles, 2 triples, struck out 92 times, had a .360 BABIP, 89% contact rate, etc. We don't have root statistics on defense like we do for offense, and that's the problem.

You can argue which sum total statistic is most accurate for offense, but the quality of the root statistics is so vastly superior to defense that all of the respected summary stats have a huge amount of evidence to support them.

For instance, why aren't we recording throwing errors and catching errors as different stats? Why when a ball that hit a player's glove but is ruled a hit are we not recording that he got to it but he couldn't make the play? It's not an error, but he got to it. We should record that somehow. Perhaps he kept it in the infield, but he didn't get the guy out at first. There's still immense value in that if there's a player on base already. Why is a shortstop getting credit for fielding a ball out of the zone when he's standing at second base in a shift when Giambi's up to bat? There's no skill in that. He stood where his manager told him to and the ball got hit right to him.

etc. etc. etc.
noseeum
Major League Manager
Major League Manager


Posts: 1697
Joined: 1 Mar 2005
Home Cafe: Baseball

Re: Defensive Metrics thread continued

Postby Big Pimpin » Tue Jan 06, 2009 6:24 pm

GotowarMissAgnes wrote:
Big Pimpin wrote:

So I pulled the data and played with LF (I only used qualified players). What I found was interesting. There is some phenomena with the 2008 data that is messing with your quick and dirty CV. It was indeed around 9% in 2008, but it was much higher in 2007 (28%) and 2006 (34%). I left 2004-05 out because it's very apparent that something changed dramatically in the measure between 2005 and 2006.

I also pulled the offensive numbers for the same set of players (I got a few more on the offensive side for some reason, probably they were listed primarily as LF, had enough ABs to qualify, but didn't spend enough time in the field to qualify in the fielding data). I looked at the same Q&D CV for BA, OBP, SLG, and OPS, and found that the ZR CV wasquite comparable. BA was around 32%, OBP was around 22% (2006 was like 38% but there were some big outliers), SLG was around 55%, and OPS was in the 33%-35% range (2006: 45%). So in general, I did not find that offensive CV were much larger.

I also had to do some research into ZR because I've never used it. I'm not sure how much I really trust it that much because it doesn't use batted ball data (with the exception of what is classified as "in zone" relative to "out of zone"). [Sidenote: The THT article talking about ZR even go so far as to say that systems like PMR are an improvement.] So say a player has an abnormal percentage of balls hit into the far reaches of "his zone" as opposed to another who has most of the balls hit relatively close to where he's positioned. The former is naturally going to have a worse ZR simply because the data drives it. Now, if you want to convert ZR and OOZ plays into a +/- system (which others have done) then you're on the right track and have something much more meaningful.

There's no doubt there are some variations across players and years, I just don't see any reason that it should be a huge worry.


I think we'll need to compare notes.

I agree that ZR is not the best measure, but unlike some of the later models, it gives a clean number that is comparable to offensive rate stats in terms of success divided by opportunities. We could switch metrics and do this more in terms of a runs driven framework like VORP or something, but the same result would occur.

As you did I only used qualified players. I am also using the Revised Zone Ratings (RZR) measures on Hardball Times. And I do the comparison position by position, because fielding statistics are so position dependent. I think you also want to exclude Manny from those other years or else use a different measure of variance than range. Range is highly sensitive to outliers, and so Manny's bizarre performance, I think, accounts for your finding on the difference between 2008 and other years (Manny does not qualify in the 2008 data). If you exclude Manny or if you use a measure of variance that is not so sensitive to his outlier status and go through any year for LF, RF, and CF, you'll find that fielding variance ranges around half of those offensive metrics.


Got it, and you're exactly right. ;-D

I guess there's still a disconnect because I don't see RZR as it stands as a metric that says anything about how good of a defensive player a guy is or even how many runs or plays he was above or below average. I'd be more interested in taking say, offensive runs created and defensive runs saved and seeing what kinds of variations there are. Just scanning through the batting runs and fielding runs (UZR) compared to average on FanGraphs, it appears there's a lot more variation in the offensive numbers.
Big Pimpin
Mod in Retirement
Mod in Retirement

User avatar
EditorCafeholicFantasy ExpertCafe WriterCafe RankerGraphics ExpertMock(ing) DrafterEagle EyeWeb SupporterMatchup Meltdown ChampionPick 3 Weekly WinnerLucky Ladders Weekly Winner
Posts: 13710
(Past Year: 5)
Joined: 20 Apr 2005
Home Cafe: Baseball
Location: Building a metric. And being ignorable and stupid.

Re: Defensive Metrics thread continued

Postby Big Pimpin » Tue Jan 06, 2009 6:32 pm

noseeum wrote:Here's a discussion on all the defensive metrics available:
http://ussmariner.com/2006/01/24/evaluating-defense/


Nice, a USSM link. That's my first web visit of the day, every day. ;-D

Yeah, Dial isn't bad, forgot about that one (probably because they don't ever use it on USSM when attempting to value players :-B ). Maybe I'll give you five. RANGE hasn't been updated since 2005, the BP stuff isn't transparent or publicly available, and the fan's scouting report doesn't really mean much IMO.
Big Pimpin
Mod in Retirement
Mod in Retirement

User avatar
EditorCafeholicFantasy ExpertCafe WriterCafe RankerGraphics ExpertMock(ing) DrafterEagle EyeWeb SupporterMatchup Meltdown ChampionPick 3 Weekly WinnerLucky Ladders Weekly Winner
Posts: 13710
(Past Year: 5)
Joined: 20 Apr 2005
Home Cafe: Baseball
Location: Building a metric. And being ignorable and stupid.

Re: Defensive Metrics thread continued

Postby GotowarMissAgnes » Tue Jan 06, 2009 7:18 pm

Big Pimpin wrote:
Got it, and you're exactly right. ;-D

I guess there's still a disconnect because I don't see RZR as it stands as a metric that says anything about how good of a defensive player a guy is or even how many runs or plays he was above or below average. I'd be more interested in taking say, offensive runs created and defensive runs saved and seeing what kinds of variations there are. Just scanning through the batting runs and fielding runs (UZR) compared to average on FanGraphs, it appears there's a lot more variation in the offensive numbers.


As I said in my pm, apologies (in advance) if I seem like I'm lecturing..it's just my job.

I agree that it would probably be better to switch to something luke batting and fielding runs and maybe I'll see if I can look at it that way this weekend (although prepping for two classes might take precedence).

But, get back to basics. Those runs data are simply built on a structure that looks at the opportunities a player had for success (plate appearances for batters, balls hit into their zone for fielding) and the success they achieved with those opportunities (whether or not they get on base or hit the ball and what happened when they hit it; whether or not the fielded the ball and turned it into an out). There is a whole range of metrics that can be used to quantify those opportunities and successes in both fielding and hitting.

RZR is probably closest in comparsion to OBP. You can create a batting runs measure based only on OBP. It ain't perfect, but because we know OBP is a damn important part of a player's offensive value, it gets us a decent way towards the truth. Similarly, RZR, like OBP, tells us how many balls that were hit into the zone are caught by that player. Like a straight use of OBP, it ignores context like park factors and run environment. Like a simple OBP based batting runs measure, it misses the outliers, so-called "out of zone" events--just as OBP ignores the importance of outliers, like the difference between a homer and a single.

So, you are right--by themselves OBP and RZR don't tell us about runs or comparisons above or below average. We can certainly do the latter by looking at how a player compares to peers. We can also translate OBP and RZR into a runs based measure, although it will not be as accurate as other translations that account for what is not there. But, they both are credible enough measures of offense/defense and an important part of any system measuring player performance.
"I don't want to play golf. When I hit a ball, I want someone else to chase it."
GotowarMissAgnes
Hall of Fame Hero
Hall of Fame Hero

User avatar
Fantasy Expert
Posts: 5516
Joined: 12 Dec 2003
Home Cafe: Baseball
Location: Happy Valley

Re: Defensive Metrics thread continued

Postby bigwords » Tue Jan 06, 2009 7:23 pm

Noseum,

You say, "OBP is the most important offensive stat. It correllates most highly with runs."

OK, you're building a team. You have to choose between Jimmy Rollins or Jack Cust (at the same salary). Who you going with, and why?

I'm interested in your explanation as much as I am with your answer.
bigwords
Major League Manager
Major League Manager


Posts: 1026
Joined: 7 Jun 2006
Home Cafe: Baseball

PreviousNext

Return to Baseball Leftovers

Who is online

Users browsing this forum: No registered users and 8 guests

Forums Articles & Tips Sleepers Rankings Leagues


Today's Games
Friday, Apr. 18
(All times are EST, weather icons show forecast for game time)

Toronto at Cleveland
(2:20 pm)
LA Angels at Detroit
(7:05 pm)
Milwaukee at Pittsburgh
(7:05 pm)
St. Louis at Washington
(7:05 pm)
Miami at Miami
(7:08 pm)
indoors
NY Yankees at Tampa Bay
(7:10 pm)
indoors
Seattle at Seattle
(7:10 pm)
Atlanta at NY Mets
(7:10 pm)
Baltimore at Boston
(7:10 pm)
Chi White Sox at Texas
(8:05 pm)
Minnesota at Kansas City
(8:10 pm)
Philadelphia at Colorado
(8:40 pm)
Houston at Oakland
(10:05 pm)
Arizona at LA Dodgers
(10:10 pm)
San Francisco at San Diego
(10:10 pm)

  • Fantasy Baseball
  • Article Submissions
  • Privacy Statement
  • Site Survey 
  • Contact