If the mix is mostly portland-based, it can result in a mix that is too stiff. Try adding lime to make the mortar more plastic.
Also, bricks laying in the sun will warm up and can wick moisture from the mortar, resulting in a lousy cure.
For strength, keep the wall moist for a few days to a week by occasionally misting with water. After misting keep it covered with a tarp. It'll slow down the cure just enough to help increase the strength of the mortar.