Raj, are you sure the relief valve isnt on the hot side? Anyhow the pressure should be around 18 psi. 2 immediate things to check are the expansion tank, it could be water logged. Next check the cold water automatic fill valve. they are factory set to fill when the system pressuree falls below 12 psi. It has a screw and retaining nut on top of it. If the screw is more than 1/2 way in back it out some and see what happens.
If your expansion tank is water logged( most likely isf it is an old style that isnt pressurized) open the drain valve on it and allow it to empty with system off. It will do a glug glugg thing until it is empty. It needs to ahve air inside of it so that the hot water( which has a larger volume than cold) has a place to go. when the tank is empty close the valve and turn on the system again.
One other possibility is that the aquastat is either out of adjustment or faulty. It is the control on the front of the boiler that looks like a box and probably says honeywell on it with wires going to it. This isa the interface betwwen the boiler and the thermostat. I monitors the boiler temp, not letting get cooler or hotter than the specified levels. The high limit should be 180-190 degreees F. If it is higher the system will over pressurize and blow off. This control unit is expensive but not difficult to install if you follw the wiring diagram and make sure to write down the wiring of the old control before you remove it. Try these things , but you may have to call a plumber.