THis is one reason, among many, that I no longer join public leagues. There is always this potential with private leagues too, but I find it happens much less often. Also, if you are picky about the number of teams, categories, number of players, etc., you can generally avoid most leagues like this. You may still get one every now and then, but I've had only 1 like that over the past 2 years (about 30 leagues total) since I stopped joining public leagues.
And once you find a good group, you generally start joining more and more of their leagues anyway.