Welcome to the cafe, leftfield! Always glad to have another fantasy junkie on board.
For panic symptoms, I recommend Dan Spazierer's hot-off-the-presses point of view article. And if that doesn't help, just think of the poor folks who are waiting for Jason Kendall to finally pick it up. You're not alone...
As to your pitching, "pretty nice" sounds like an understatement (assuming that Johnson is Randy). That's quite a group you've assembled, no matter what your league format is. Yes, you should be very worried about Brown, but it looks to me like your staff is good enough to survive even that loss, should Brown miss significant time. Sure, you have a few question marks, such as D'Amico's durability, Beckett's inexperience and the ever-popular "which El Duque will show up today" guessing game, but on the whole, this rotation is as good as they come.
Duckworth, of course, is very good as well, so picking him up isn't really panicking. If Brown goes on the DL, you may be able to pick Duckworth up for free, but even if he doesn't, I'd strongly consider dropping D'Amico for him. It's a judgment call, but with your starters, I'd lean toward the slightly safer option.
Hang in there, and don't get bitten by the panic bug just yet!