Are prospect lists a crapshoot?
I'd say you need to know how to interpret them:
1. Those who make prospect lists tend to overrate AND underrate players under the age of 21. Simple reason is that there isn't much professional experience to make a reasoned determination how good these very, very young players actually are. So you can look back and see players who should have been higher and players who should have been lower...more often than not, it's due to the age factor.
2. Prospect lists tend to overemphasize "upside" and underemphasize "certainty." Because these players are very young, scouts generally just pay attention to "tools" -- whether a guy has power & speed -- and assume that eventually, the players in question will learn how to use their raw abilities. Players who have less tools, but have shown the ability to do well anyway thanks to plate discipline or throwing control -- tend to be overlooked.
3. Defense plays a big part of these rankings. When we come up with fantasy rankings, nobody pays attention to defense...but here like in real baseball, it matters, and it certainly skews the results.
4. For fantasy purposes, you need to look at how close these players are to the majors. Age, highest minor league level attained so far, organizational opportunity, organizational track-record for promoting prospects, and demonstrated success (not just hyped potential) should be measured.
5. Those who do this thing are in a small club (though the club is getting bigger with the rise of the Internet). Everybody reads each other's work. Usually, rankers like to play it safe and not look foolish. But everyone has a pet prospect and a predicted dog.
All in all, I'd say say these rankings do a very good job at predicting future major league stars, but you can't just look at it as a list of "this guy is better than that guy..."