I would first consider power washing the siding to remove dirt, grime, and some of the chalk.
I use a product from Sherwin called A-100 that is a good quality paint with primer material mixed in. It is good for your application and for painting brick and cement.
Check it out.
© 2016 Renovate Your World LLC