Hi Dennis. You have a problem, you have just explained to two types of fungus,. ( you cound have both,) 1) which is common, in damp humid areas, and no movement of air, ( little fibres and black.? ( Wet Rot) Coniophora cerebella. this only attacks wet wood and will stay on the wood, it will not go any further than the wood its on. some people call it mine fungus, it will only rot the wood. 2) the one I hope you have not got, (Dry rot),Merulius Lacrymans. again very common in your part of the world, you say it brown and theres a kind of sand, the sand type stuff could be the basidis ( sporophore ) seeds from the the brown stuff,(( Fruiting body) it will land on wood and start again, spreading itself looking for food,( wood) the brown stuff looks like pancakes, you should see some fibres strands around called mycelium ( gray/white in colour) this feed the main body.
the area, but be dried out try to get some air moving around ( but befor you do that clean up all the sand type stuff ( spores) yo dont what them over the house or it will start again also clean your shoes/boots, make sure youre not tracing it around the house.
now you know what you've got do as Steve has said, contact the authorites, come back to us on this, please, others also have this problem and they think oh, it will go away "wrong" John 2000