We had a similar problem in our house. I'm assuming here that, like with our place, you only hear the noise when the heat pump outside unit is running.
The coolant tubing that run between the indoor and outdoor unit in our house were run fastened to the floor joists in our basement. The larger tube was wrapped in pipe insulation, but the smaller tube was not, and was laying directly against the floor joists. When the unit was running, the vibration was transmitted from the tubing to the floor joist (and anything connected to it) all of which acted like a huge sound board.
I would inspect this tubing from the point it enters the house to where it connects to the air handler, and see if there is any metal-to-metal or metal-to-wood contact. If so, insert a piece of foam pipe insulation to damp the vibration. Or, just for testing, even a rag will do.