Josh, I'm assuming that these 2nd floor beams and joists are closed over on the underside, meaning your first floor has a drywall or plaster ceiling, rather than exposed beams?
If the beams/joists are exposed, you should shim the floor above directly between the joists and subfloor, shimming only as needed on individual joists. If the joists are not accessible, that's another matter.
I once successfully leveled a large, 103 yr. old. home which had a belly nearing 6" in the center, but before I did I had a structural engineer come look at the home and my plans for repair to insure I wasn't missing something critical. This might be a good idea in this case, too.
I'd certainly look into getting things level without adding another plywood layer if possible. Adding another shimmed plywood layer would mean you'd have to use at least 5/8" ply for rigidity, plus your shims. That means you'll probably be cutting down doors, casings, etc., and your top stair riser will suddenly be higher than the rest, making a place where people will trip because the rise is not equal to the rest. good luck!