I think the refurbs can be a bit of a crapshoot since you really don't know why it was refurbished. A lot of the time the manufacturer has "repaired" the phone but it still doesn't work properly and then you get stuck with a nightmare that's more trouble than it's worth. On the other hand, some of the phones may be refurbished just because someone didn't like the design and exchanged it for a different one, depending on the store's return policies so you could luck out and get one of those ones. If you can get some kind of gurantee or warranty I would do it. Otherwise, if you're a gambling man, just buy it and pray.
If you're spending any significant amount of money though, I would try to get some kind of guarantee on it if possible.