The "Luck factor" is completely dependent on how good the other players in your league are. If you are playing in a league with experienced veterans and everybody spends as much time researching and studying as you do, luck is a MUCH bigger factor. This is true in both football and baseball. Even so, I still believe there is much more luck involved in fantasy football. A poor fantasy football player can still win a league full of skilled players if they draft a breakout player like LT2 was last year. A poor fantasy baseball player, on the other hand, has almost no shot at winning a league full of skilled players. As Knapp said, you are starting twice the players in baseball than you are in football which means that each player's impact on your baseball team is roughly half that of your football team. You might get a few lucky picks, but that luck alone won't carry you through a 162 game season.
Contrary to what some people here think, playing fantasy sports is not chess. The "skill" ceiling in fantasy sports is nowhere near as high as some people claim it is. If a player experiences success time and time again, the odds are pretty good that the leagues they're in aren't very good.