The foundation you describe sounds very similar to that of the house where I lived as a teenager (built in 1905) in NE New Jersey. On the outside, the stones were nicely aligned and pointed up, but on the inside it was as you describe, with the stones nearly covered with a very crumbly cement.
What I discovered was that the walls, though crumbly on the surface, were quite strong and showed no signs of cracking, shifting, etc.
The real issue seems to be whether there is any shifting or cracking of the foundation. These old foundation walls were 16" thick (compared to 8"-10" today) and *very* sturdy. I think they deliberately used a very sandy mix for the mortar so that any shifting of the foundation would be distributed over the whole wall, thereby reducing the likelihood of cracks.
If the outside of the foundation wall seems to be crumbling, or if there are any significant cracks, this will probably require professional repair, but if it's just a case of crumbling in the basement, IMHO it shouldn't be cause for worry.