I normally don't pound them back in because its a short term fix and they will waller back out again. You can try to remove the old nails without damaging the surrounding rock with the hammer and then screw a longer drywall screw into the existing hole and then spackle it over.
The screw needs to be longer than the removed nail.
Another method that I use is to pound the nail back in then drive a screw above and below the existing nail, counter sinking it into the board. the screws will keep the board and nail from moving out also.