"I'm not even sure what the deal is with Crow. Is he a closer?"
Soria is back in the closer role with Crow back to 8th inning.The best holds guys are the 7th and 8th inning guys on winning teams and as we've seen KC hasn't had many late leads to protect lately. Crow would be the easiest to replace
"I try taking either saves, holds, wins and Ks each week"
The weeks where the SP have tough matchups go with holds guys and avoid the good hitting lineups or facing ace SP's. You should be able to add a decent holds guy from the free agent pool whenever you need 1, look for 7 game weeks not facing the best hitting teams.
When asking for roster advice in these pages, remember there are thousands of different sets of league rules & many different active roster settings. If you want the best replies, be sure to include your leagues particulars in the original post