The HUGE yellowjackets could be carpenter bees. If you see them 'hovering' a lot, this is usually an indication that they're looking for other bees invading their 'territory'. If you want to know if you have carpenter bees, look on the ground or on railings or on roofs under the soffit(s) to see if you have saw dust. Sometimes you can hear them knawing away at the wood. Usually, carpenter bees work their way UP into the wood then TURNn left or right for a short distance. Then, the egg is laid. Poking into the hole isn't of much use unless whatever you're using is able to bend 90 degrees as you push it in. I recommend that you get some white, latex caulk and fill the entire hole, bee and all (if she's in there), and the next time you paint, you'll make it look nice.
Their bore-holes are about 3/8 - 1/2" across. They're really nice and round too. If you have any gaps in the home that are larger than the aformentioned size where bugs can get into the home, then they may be inside too. Places like the attic or up under the eaves are popular places for them. Good luck and hope you don't have 'em.