I used to have this problem in a few electronics hobbies. I'd try a bit of electrical tape rolled up and squished flat. Put that behind the battery tab or spring.
Pro-tip:. Buy a roll of white 3M electrical tape. It's way more expensive, but the extra money is worth it. You can see it easier if you drop it in the battery compartment, and it doesn't leave black goo everywhere.
--Matt