It's very hit-and-miss, sure, but not completely random by any means. I did a quick study on the active guys who were HoF locks or would be within a couple of years at career rates of production. The vast majority were first rounders, and there were as many who were first overall picks as were drafted after the second round.
a 50% success rate in terms of having major league careers for first-rounders is still, far, far better than for the rounds after it. You can point to Pujols, I haven't checked but I bet he's the only guy to have completed a full MLB season for several rounds either side of him.