From the sound of things, sandpaper is not the way to go. I thought that you might be able to scuff up the surface to make the cleaner work faster but if it is taking the paper off the drywall, then it is too much. You have to leave the gray paper on the drywall.
As I recall, there was a contact cement years ago that was blue or green and that might be what you have. I don't know of any solvent for it that might be safe to use indoors. Sorry.