I don't follow what you mean by replacing the whole window? I presume you are referring to replacing just the sash. By replacement windows I mean the entire window.
It makes more sense to replace the windows first. In that manner when the siding is done the installer can trim around the window the best making it look like a new house rather than a makeshift trim application.
I good quality vinyl window can be made to size cheaper than most wood windows. Use a reputable company and get 3 bids. Use a good quality siding product also not some low price junk that most will try to push. If you want an absurd price call the company that *** **** represents.