Someone else was discussing the same thing a few days ago, i.e. getting rid of all their closers in a 5x5 H2H league. I can't remember who it was that pointed this out, but they made a very valid point. If you punt saves you automatically start out 0-1. Which means that if you won 5 out of the other 9 remaining categories you would only tie 5-5. You'd have to win 6 of the remaining 9 to go 6-4. That can be hard to do.
If you keep your closers then they at least give you a good fighting chance of getting a win in a category, plus they'll normally help in the ERA and WHIP more consistently. That is they can't get shelled for the same amount of innings as a starter could. A starter going 6 innings and giving up 6 runs hurts more in a way then a closer coming in and giving 2 in one inning pitched. The logic behind that being that it's easier to bring a high ERA from 2 runs down then it is from 6. The same can be said for WHIP of starters and closers. And every starter no matter who he may be WILL get rocked every now and then. Plus they'll probably get hit hard a little more frequently.
I'd stick with what you got going and wouldn't do anything drastic. You're winning anyways right?
I hope that comes across the way I meant it