Normally when a fence is constructed the unfinished or post side of the fence faces the property of the person who constructed it. This is purely cosmetic particulary so with a wood fence. However I don't know of any codes that says it has to be that way. A few years ago I constructed a privacy wood fence around my back yard which contains a Pool. The fence on one side of my property had the finished side facing towards my property. I naturally thought the fence belonged to my neighbor. Upon applying fo the building permit I was informed that based on a copy of my survey the fence was on my property and was considered mine. Even though I didn't have to, I still talked to my neighbor about it before I replaced it. This is a grey area and you need to talk to your neighbor, and if they object, talk to your local Building Inspector.