Chris, you're right; getting rid of all that glue can be a real headache. I know you'd said you've washed with lots of things...was one of them a warm water and white vinegar solution? I've found this to cut the paste better than anything else available.
If you can avoid it, I'd try to not use the skim coat method, as it would be a tough job to get right if you don't have much experience in working with it. You may need to skim over places where your paper scraping has gouged the face of the drywall, but I'd try to avoid skimming the entire walls unless absolutely necessary.
One good option would be to paint the walls with Zinsser wallpaper prep. This is a sizing normally used before repapering, but it seals onto existing glue and drywall very nicely, and it can be lightly sanded very easily and your wall is then ready for paint.