Since most of the time whenever I see a penny its either in the street or some other heavily-trafficked location, I won't bother to pick it up. I have enough pennies at home that I've rolled up and have yet to take in to the bank.
The only times I'd pick one up is if it looked reasonably clean and was just sitting carpet or a tabletop or something like that.
I must admit, though, that I'll occasionally "take a penny" when I need to make change at a restaurant/convenience store. And who KNOWS where those pennies originally came from!