Hi, Erin. The problem is not with the poly, it is the nature of wood and humidity. Wood expands and contracts with changes in humidity. Wood can easily contract 10% across it's width. It moves in length, but a lot less, and thickness, but this is hard to see. If you fill the spaces now, when the wood expands in the summer, it will crush what is in the space and the wood itself, leaving a bigger gap next winter. A not too comforting thought is that your wood furniture is doing the same thing and this is the reason that your expensive dining room chairs will eventually come apart.
The solution is easy. You need a humidifier in the winter. This will keep the humidity in your house constant and be more healthful for you. You want a humidifier that you can control to keep about 40% relative humidity in your house.