If you're using thinner on the brushes, you're using oil-based poly. Oil-based poly will make the grain stand up as compared to water-based poly, hence, a possible source for the bumps. Oil-based is 'harder' than water based so a couple or 3 coats will be necessary along with your pre-stated sanding. Did you shake the can instead of stirring it? You need to stir the solids 'out'. They can be your bumps too. Shaking the can won't mix the poly properly. Are you sure you're sanding enough, again, because of the raised grain? Just some thoughts because other than that, you're doing the right thing in my mind.
Henry will be by shortly for more (I hope). My best to ya and hope this helps.