A quick lunchtime hunt at a old homesite, out in the middle of nowhere, gave up my oldest coin so far. 1839 seated liberty half dime. Not a pristine example but I'll take it.
I was told that it was common to have coins with holes punched in those times. It was done to lessen the chance of losing or being stolen by stringing the coins to your body. No idea how accurate that is though.