the outer most cardboard skin of the drywall is for painting, texturing, and any thing will adhere to that skin. Without that skin, you chances of any material including paint to adhere is greatly compromised.
you might try floating out the bad areas where the skin is removed and try that. Sand smooth to the level of the skinned board.
You might be pretty close to re rocking the bathroom with 1/4 inch drywall.