Milk Paint is, IMHO, one of the worst paints to try to remove. Most methylene strippers barely even soften the stuff. On a porous surface like brick, I'm betting that it will be next to impossible.
I'm hoping that Hy will chime in on this, he seems to be the expert on "Peel-away" which is a sodium hydroxide stripper (which I've never had the opportunity to try on milk paint).
This is my guess; apply a sodium hydroxide stripper to your brick and allow it to do it's duty. If your using a liquid form such as a deck stripper, make sure it stays wet for several hours by lightly misting it with water (Peel-away stays on for 24 hrs). Then use a power washer with at least 2500 psi to clean off the residue. If your brick is in good shape, you should be able to be fairly agressive with the washer.