Grabban, who followed manager Andy Scott north from Brentford this summer, struck home in the opening stages of the second half with the outside of his boot from 20 yards and Oxford had no answer, despite a late bout of pressure.
Chris Wilder's side are among the favourites for promotion this season, though, and they lived up to their billing in the early exchanges with Asa Hall hitting the post from Damian Batt's free-kick and Michael Duberry criminally hitting the rebound over from close range.
But they could not capitalise and Scott's side made the perfect start to the second period as they needed just three minutes to take the lead, with Grabban finishing with aplomb.
Peter Leven had Oxford's best chance of finding a leveller when he shot wide after breaking free from a static Rotherham defence.
But the hosts held on and could have doubled their lead but Ryan Clarke superbly denied Grabban.
Any reproduction, publication or redistribution of this material without the written agreement of 365 Media Group is strictly forbidden.