Look at it from this angle: When you have it completed AND during construction, you want to be able to insure it. Homeowner's policy. To be insurable, it must be built to the building code in the area and inspected.
When you see a dense tract of homes it is not because of building code restrictions but because of the cost to the builder (which he must pass on to the buyer). And when you hear of restrictions on what you can and cannot do to your place, you are talking of homeowner associations, which you will not have in building as you outline.
Here in SoCA a building permit is usually valid for six months; that is work must be started and the first inspection must be within six months. The other inspections can trail off after that. The idea is to assure that the latest codes are being followed. If the permit expires, it will have to be reviewed and reinstated. This is not the end of the world, just expect to do it. That is here; your situation may be different. You will have to ask the building dept.
As to getting a hard time from the building dept, you will need a set of plans that will get the permits, and an architect is the best starting point. An individual home should be no more problem than a tract of homes, unless you are trying to do something weird. On sparcely developed land, you may run into environmental restrictions; that may be your major problem. You should see what a tract builder has to go through here!
And a word of caution: There are several envirnomental activist groups that oppose development of any kind; their usual method of expressing their dislike is to set a match to it. One such in the papers here the other day was an ELF member that torched several SUVs in dealer's parking lots.