... It would be prudent to hire a Home Inspector or a Termite Inspector to look at the problem. A Home Inspector, IMO, is best because they SHOULD know about construction. A Termite Inspector may NOT necessarily know anything about renovating a house. At worst, get a couple more opinions from other general contractors. IF the damage is as bad as you (and the contractor) say, should you be living there thinking that the walls may come crumbling down at any time?!
My best to ya and hope this helps. If you need any more 'direction', post up!