I will assume the unit is a boiler as they were installed with asbestos.
(Furnaces which are used for warm air were not usually wrapped with asbestos. The fire chamber of a boiler was called a furnace back then. The unit is definitely not a stove because you don't want to cook on an oil-fired stove.)
You don't have to remove the old boiler and its insulation unless there is no room to do so. You can "abandon", seal it in place, and enclose it to prevent asbestos danger which comes from dry, loose particles entering the body. Asbestos removal can be expensive.
The piping can be arranged to connect to a new boiler. Your choice, per jb and jj, is whether you want to pay for a new boiler to save money in the long run - or not buy the house.