I disagree with the consensus of opinion. Let's break down your categories:
Wins - Edge: Starters Modest Contribution: Middle Relievers Small Contribution: Closers
K's - Edge: Starters
ERA - Edge: Relievers
WHIP - Edge: Relievers
Saves - Edge: Closers Small Contribution: Middle Relievers No Contribution: Starters
Now the suggestions so far:
Losses - Edge: Relievers
Holds - Edge: Middle Relievers (exclusively)
IP - Edge: Starters
QS - Edge: Starters (exclusively)
K/9 - Edge: Relievers
Total Bases Allowed - Edge: Relievers
Now all of these are great categories in the abstract but only the IP and QS suggestions help you balance out your reliever-friendly setup. Losses and Holds, although great categories to work into the right balance, would completely blow away any necessity of a deep bullpen. In that league, I'd try to draft Santana, Peavy or another top-flight pitcher, and then all relievers, basically.... maybe pick up another starter or two if someone looked really hot but as long as I was competitive in K's and not last in Wins, why bother blowing up 1st place in everything else?
You definitely need a SP-friendly stat to avoid the pitching in your league to turn into who can scorn the most starters.
0-3 to 4-3. Worst choke in the history of baseball. Enough said.