Look for dirty spots in your insulation, which often indicate holes where air leaks into and out of your house. You can seal the holes by stapling sheets of plastic over the holes and caulking the edges of the plastic.
Check your ducts for air leaks. First look for sections that should be joined but have separated and then look for obvious holes. Consult with a professional about repairing duct leaks. Safe duct repairs require a licensed heating, ventilating, and air-conditioning contractor.