The visitors when ahead in the 13th minute when Johnny Mullins turned the ball into the net at the far post from a corner from Sam Wood. Ten minutes later Alex Revell beat Hereford goalkeeper Adam Bartlett with a low drive from 25 yards to double the Millers’ lead.
But Hereford reduced the lead in the 34th minute when Yoann Arquin latched on to a short pass from Delroy Facey before turning a 25-yard shot into the roof of the net.
And the comeback was complete three minutes before the break when Delroy Facey took advantage of a mistake in the Rotherham defence to beat Conrad Logan from close range.
Straight after the break Brett Williams had a chance to restore Rotherham’s advantage, but his close-range shot was well saved by Bartlett. Hereford also had chances with Tom Barkhuizen driving a free kick over the crossbar, while Arquin shot inches wide from 25 yards before the Millers struck again in the 65th minute.
Mullins latched on to the ball 10 yards out before drilling it past Bartlett to seal the win.