You need to fill the jamb. Sometimes you can find molding just the right size (1/2" thick) to fill the gap. You may need to rip (cut lenghtwise) to get the correct width. take you measurements and go check out selections available. If you have a table saw, you can cut the from any piece of lumber. Sand, then paint or stain to match.
Shaving the drywall will not solve your problem because the casing will tilt inwards leaving a gap on the outside, and the corners won't match.