You didn't mention anything about a basement or crawlspace. These 2 are the majority sources of where water enters the home and creates all kinds of problems, one of which is mold and mildew. Assuming you have one (or both, which I have both in my home), long-standing water isn't necessary to start this process. As water evaporates, it needs to collect on something unless it's actively removed before it has a chance to evaporate. If you have a crawlspace with either dirt or concrete, simply lay down a sheet of 4 mil plastic that's about 4' square. Tape or weight the sides down to trap the moisture for your test. If after a couple of days or so, you see moisture on the underneath side, well, you've found 1 source of the moisture -> mold/mildew 'smell'. You can take this step further by duct taping the same plastic on the wall(s) of your basement/crawlspace (all 4 of them) to see if there's moisture transpiring there. (And be liberal in the number of plastic 'squares' you put up. You're NOT limited to 1 per wall.) Be sure to tape ALL 4 sides of the plastic so you trap any moisture on the underneath side. In all cases, you don't need to wait for a rain. If you have moisture on a dry day, chances are you'll have MORE on a wet one. If you find that doing this type of testing doesn't fruit any results, start looking 'up' to see if there is any residual mold/mildew in the floor insulation, starting in the basement & crawlspace. It could be on the joists or underneath-side of the sub-floor as well.
If all your efforts prove fruitless, well, post a follow-up to your original post, summarize what you've done, and we'll go from there. If you do find your tests fruitful, then again, post a follow-up to your original post and summarize what you've found, and where and how.
My best to ya and hope this helps. Here are a couple of questions you want to answer. 1) What are the dimensions of the basement and crawlspace l x w x h? 2) Are they dirt or concrete floors? 3) Are your gutters free & clear of debris? (Go out with an umbrella in the next heavy rain. This will tell you if the gutters are clogged AND properly angled.) 4) Are all your downspouts draining the water at least 3' away from the foundation? 5) Do you have gutters on all the eaves? 6) Is the landscape around the entire foundation of the home angled away from the house sufficiently?