Glenn Loovens volleyed in Barry Robson’s corner after 14 minutes but the home side squandered a series of chances.
The frustrated home fans did not rest easy until McGinn, on for Scott McDonald on the hour mark, fired a wonderful drive past Tomas Cerny.
It was not the best performance of the season from Tony Mowbray’s side but the win puts pressure on the Ibrox men who play their SPL game in hand against Hibernian on Sunday.
The home side opened the scoring after Robson’s cross from the left found the Dutchman 12 yards out and he sent a left-footed volley past Cerny.
But the Hoops had to wait until injuty time before they sealed the win when McGinn fired in a wonderful drive from 20 yards.