You get a reading of 24 volts from these terminals because the thermostat switches are not closed between them. The R termonal does have one wire from the transformer, but the other terminals have electricity through coils in relays; they will not supply 24 volts to a humidifier motor.
Usually, furnace manufacturers supply a transformer with enough power to run the furnace, but not another machine like a humidifier at the same time. The adapter you have is a transformer to take 115 volts from a receptacle and convert it to 24 volts for the humidifier motor.
It is best to buy a transformer that you can wire into the 115 volt power, like a Honeywell AT140A1000 after the power switch at the side of the furnace to supply 24 volts to the humidifier. Then you can cut the plug-in transformer out of the humidifier wire and connect the wires to the 24 volt screws or wires of the transformer.