Justin, I think you need to scrape off what you can with a floor scraper. You might talk to a floor tile company to see what they have. I talked to some guys recently who were retiling a hospital hallway. They told me of their trials and tribulations getting a couple layers of old tile off of the concrete. They indicated they put down some type of solution/stuff on the floor afterwards so new tile would stick. They had a fairly smooth surface to work from when they put it on. I don't know what they used to get the old stuff off or what the new stuff was.