How to get stick stuff off walls?

I had glow in te dark stars on my ceiling and I am repainting my room. I took the stars off but the little white,spongy, circular sticky things wont come off. I tried that goo gone stuff and I didn't work it just made it slimy. I don't want to paint over them because I don't want chunky stuff on my walls. When I try scraping them off it also takes chunks of drywall with it. Any tips on how to get it off without ruining my walls? I don't care if it ruins the paint because I am just painting it when I get the things off.