For what you're doing, I suggest that you fill in a few inches of dirt at a time and use a Tamper. If the hole is too small at the start, use a piece of 2 x 4 or a piece of 4 x 4 until the 'level' in the hole rises sufficiently to use the Tamper. There are different sized Tampers. A 12" one should be plenty. (If you want to use a 2 x 4 or a 4 x 4 for the whole job, that's fine too.)
Although you're most likely on top of this, I'll add a few extra comments for others. Don't fill the hole(s) with gravel, to only top it off with some dirt. This will permit water to fill the air space in the gravel, thus, collecting in the hole and most likely making its way to the foundation (and into the house). ONLY put a few inches of dirt in the hole at a time because if you put in too much dirt and compact, you may create pockets of air due to an excessive amount of dirt at one time. The air pockets that may be created will fill with water and the aforementioned will occur too. The minimum slope is 1/4" per foot for at least 3'. The more the merrier but the key is to have slope for at least 3'. If it's less than 3', it's worth considering 'collecting' it and moving it further away from the home. With a minimum rise of 3/4" over a distance of 3', a heavy rain could lead to water collecting and rising and making its way to the foundation, especially if the ground is saturated or drought-dry. (And we've had BOTH in our area just this past summer!)
As a general rule, I practice planting trees and shrubs so that they're about 2-3' from the foundation when they're full grown. So before planting, determine the best-approximate radius of the plant, add 2 to 3' to that measurement, and that's where you plant the plant. For example, if you want an Azalia bush to have a radius of about 3' when it's full grown, then plant the bush at least 5' from the foundation. (3' for the bush's radius + a minimum of 2' away from the foundation. The bush will have a diameter of 6' in this example.) Airflow is critical around the foundation even with proper slope. Obstructed air flow promotes mold and mildew and such.