Hi, Jim. Yes, you do have to remove all the flaking paint before you repaint or it will just flake again. The wood being wet when it was painted or just cheap paint are among the possible causes. Even painting on a real humid day can do it. Since you are doing it now, you must be somewhere pretty warm. When they did it originally, it could have been so hot that the paint dried without having time to bond also.
I would start with a wide scraper with a replaceable blade. A Hyde model 10550 is a good choice and not real expensive. You might want to follow with either hand sanding to feather paint that is sticking into the flaked areas or use a finish sander. When you have about finished a days worth of scraping, put a coat of primer on the area that you have done. This will protect the wood until you get the paint on.