We had a similar experience when removing 30 year old wallpaper. Part of the drywall "plaster" finish was left, but in most places a thin layer of paper was torn off with the wallpaper, leaving a smooth layer of brown paper in its place. My friend, who happens to be a plasterer, told me to use a thin layer of joint compound (spackle would take too long) for a large area) over the bare spots. If you don't, moisture can soak into the drywall and cause problems later. I talked to a Menard's employee and spent the money to get the proper application tools. It's tricky stuff to work with at first, but I eventually got the hang of it. Then you sand it a little (a damp sponge is recommended) then repaint or wallpaper.