Stan, you will want to get all the adhesive off. It should come off with water but an agent to help with this will make the job easier. DIF or a similar product from the home center is what you want and be sure to follow label directions.
Anything under the paint will show up on the surface. Washing with hot water is easier than sanding. It will be easier to clean up after washing also. Some wallpaper adhesives get pretty gummy and you might go through a lot of sandpaper to get the same results you would get with a sponge. Better for the "health" of the walls too.
Don't forget to prime before you paint and choose a paint with a lot of mildewcide.