I'm not sure what is happening without being able to see it. If you washed well, paint should stick but not as well as if you had put a primer on first. What brand of paint did you use and oil based or water based? Is the paint peeling all in one area or all over?
My guess is that differential expansion and contraction might be the cause. Your walls move with changes in temperature and humidity and the paint is supposed to move with them. Todays oil paints will allow a little movement and water based will allow more, so you can put water based paints over oil based paint but not the reverse.
You might take some of the flakes to the store where you got the paint and ask them to send it in to the manufacturer for analysis. They will blame the problem on everything but their paint but it might give you some clues.