Jason, are you leaving a cover plate off? There should be a sheetmetal shield over the access to the pilot or the whole area with the control valve. This is typical of the ones I have seen anyway. Leaving this cover plate off can create unusual draft situations where when the burner shuts off the remaining heat continues with a strong draft from the heat rising that blows the pilot out. If you are leaving the plate off, "Since I am just going to have to remove it again anyway" this could be making the problem worse.
You probably need to clean the pilot burner since soot and other buildup might be affecting the amount of flame. I'll leave it to you as to whether this is in your area but remember that you can damage the pilot orifice pretty easily.
A pro will probably have some other thoughts but I would certainly pick the safest choice as far as working on it.