Box in the windows with 2by6/8/10 treated lumber depending upon the thickness of the walls. Use wider lumber if you have to go thicker to match up with an inside stud wall. Use concrete tapcon screws, anchors, or a power nailer to attach them. Besides the 2by framing allow about an inch extra (more or less depending upon how good and straight and level you can cut the openings) on the sides for shimming. Basically take the overall measured width of the window(outside of frame to outside of frame,not nominal), add an inch(give or take) for shimming, and add the thicknesses of both of the 2by boards(window plus 4+"). I've found a lot of treated stuff is thicker and measures 1 9/16" or 1 5/8" thick(in other words, don't assume 1 1/2").
Make sure you buy the windows and lumber and have them on site to get actual measurements before cutting the concrete. For top to bottom, Add an inch for shimming on the top to bottom overall measurement and use a treated 2by footer the same size as the sidewalls. The header above may need to be a double 2by8or10. You may need to build a box for the top header for your sills to sit on since the basement wall is thicker than a standard stud wall.
I assume you are going to hire a concrete sawing company to saw the windows or possibly rent one if you can find one with a frame that will mount on the wall and make a straight cut. The ones around here that do that get about $175.00 per window. Maybe you can get a volume discount. don't forget that they have to be a certain height off of the floor or less and that number and formula for egress windows escapes me now but where you bought the windows, they should know.