All I can do from here is suggest some things to look at.
1. How old is your water heater? If your water heater is beginning to rust out, or if it is full of deposits and needs a good flushing, you could be getting stuff from that. This would make sense if you only use hot or warm settings occasionally, such as when you are having this problem. 2. Is your water city water or from a well? If it is from a well, you may be experiencing excessive minerals in your water supply. 3. The transmission on the washer may be leaking some fluid into the tub, although I doubt this. If your washer is a GE, SEARS (some) or Hotpoint, you can call GE at 1-800-626-2000 and they would be happy to help you 24-7-365 days a year. 4. Do you have kids who like to put things in their pockets? My 7 year old left a red crayon in his pocket the other day. Oh My Gosh, what a mess! 5. The outer tub of your washer or one of the components in the water system of the washer may be rusting out. Check the tub, the basket (the part you see when you open the lid is the basket) the water pump, and your hoses and connectors.
Hope this helps or at least sends you in the right direction. If all else fails, see if a tech at a local repair shop has some ideas. They see stuff like this every day.