A friend of mine had a disagreement with his large suburb building inspector and chose to battle. He won the case in court but his house was reappraised soon after and his taxes went up significantly. Anytime an inspector saw him doing more than raking leaves, he stopped by to see if it was work that required a building permit. It got so bad that my friend could hardly drive a nail to hang a picture without having a visit by an inspector. He ended up selling the house and moving to another suburb.
I'm not saying that is what will happen, but it could.