It's gonna be a dirty job but there are a few things you should address.
Wrap ALL your piping in the crawlspace. Measure the the width and length (as best you can) and go to the Home Center and get the proper pipe insulation. Then, install fiberglass batt insulation in between the joists, paper side up. Then use either chicken wire to nail to the joists underneath to keep the insulation from falling out. (Those joist hangers are no good in my book.) Or, what I did was pound a bunch of tack-nails about 1" long about 3/4ths of the way into the bottom of the joists, and ran nylon string around them, from joist to joist, nail to nail, to keep the insulation up. (I kinda made like a spider web.) In the space you're talking about, it might be difficult to work with chicken wire.
At the 'ends' of the joists (what's called the ribbon board), cut small pieces of the fiberglass insulation, and 'stand' them up on the ends, paper side in. Don't close off your crawlspace because you'll force the humidity into the house. (If it's already closed off and you aren't having any problems, let it go.)
If you need more info, post up. My best to ya and hope this helps.
PS: Use something like R-30 or R-36 insulation. Measure the 'width' of your joists because your insulation should fit 'comfortably' in the joist cavity. The insulation packaging should tell you what type of 'cavity', or space, the insulation can be used in. You see, if you buy an insulation that's waaaay to thick, and then you compress it to fit, you've defeated the use of the insulation. It needs to be 'fluffed'.