I suggest that you cut the hole through the brick.
Mark the size of the hole on the outside of the brick wall.
Buy a long carbide tipped masonry bit (they work best in a hammer drill) and drill through the wall around the perimeter of the marked hole. After you have drilled at least 8 holes through the wall(preferably more), use a masonry chisel to knock the plug out of the hole and smooth the sides of the hole. You may have to drill some holes across the center of the plug to get it out.
Make sure that your drill holes are perpindicular to the face of the wall, and first check the other side of the wall so that you don''t drill into electrical wiring or gas pipes!
It is easier than it sounds. Most old brick is easy to drill.
After your hole is drilled and cleaned up, insert and caulk the dryer vent exhaust hood in place.