Don't pull up the plywood. To insulate the floor you can by batts (rolls) or blankets (strips) of faced fiberglass insulation. Install the insulation with the facing (vapor barrier) towards the warm part of the house. Before you start, you need to know the spacing between the joists so you can buy the right size material. Insulation is avialble in 15" and 23" widths to fit between the joists. Also be sure to get the support wires that hold the insulation in place. Don't know your climate, but R-11 is minimum for floors, and R-19 is about the most anyone puts in.
You can put plastic sheeting on the earth and overlay it with some stones if moisture is a broblem in the crawl space. I would consider this optional in dry spaces.