There is an art to doing seams and joints when it comes to drywall.
Usually, seams are feathered outwards (from the seam) with successive coatings of joint compound over a number of days. This way, the transition from one sheet to another isn't as noticible. It's hard to say what you should do. At first thought, I say remove anything that's loose and visible. Then again, maybe all you need is another coating or 2 or 3 of compound to 'fix' the problem.
What I do suggest is that you try to get a pro to do just the 'repair' part of the job. If it involves ripping the 'old stuff' out, so be it. If it involves just skim-coating more joint compound, so be it. DO ask the pro what they use for sanding. If you're NOT prepared for a lot of dust, you'll have a large mess to deal with. There are sanding 'tools' (both professionally made and home-made) that will suck up the vast majority of dust as the dry mud is sanded. It would be worth having a pro do this as 'everyone' will see the finished walls and ceiling.