Jim is most likely right on the water heater. He brought up the 5th possibility which is the pressure tank(assuming you have one). I have seen some of those sit in damp basements and and out in well houses and look so rusty on the outside that I hate to think what the inside looked like.
Do the tests like Jim explained and see if any of it comes from the cold side. Post back with what you find out. It always helps to have feedback. Thanks in advance.