I'm a pretty big believer in the H2H points system, as it allows for some randomness (h2h) but also doesn't hold some of the typical categories to higher value than what is actually important in baseball (SBs, saves as examples).
When I designed a system for an ESPN points league, the first thing I did was develop a point system for the hitters, because this is a bit easier. One point for a single, 4 for a homer. 1 point for an RBI, .5 point for Run and 1 point for a stolen base, since it shouldn't be more important than a single IMO. So this game an approximate amount of "points" that the hitters would be worth.
Armed with that number, I was able to look at pitching. Since I value things a pitcher can control (strikeouts), I put that at a point. And since I think a 200 inning 4th starter is more important than a closer, I wanted to make each out recorded a half point. Because I value 200 innings more than 60, I wanted to avoid rate stats common in roto. At that point, you can come with arbitrary values for things that you and the league value, like complete games, quality starts, saves, wins, etc. It's not impossible, and over time a good system can be developed. I think it's important to get league by in though, so showing your work is always a good thing.