Vale were quickly on the offensive, Gareth Owen having a close-range shot saved by Hereford goalkeeper Adam Bartlett. But Hereford responded strongly with Delroy Facey, Simon Clist, Joe Colbeck and Yoann Arquin each going close.
The visitors, however, rallied and finished the half strongly, Guy Madjo drilling a low shot at Bartlett, who soon dived full length to parry a 25-yard drive from Doug Loft.
The deadlock was eventually broken 10 minutes after the restart when Tom Barkhuizen cut in from the right to lob the ball over Vale ‘keeper Stuart Tomlinson from 18 yards.
Vale made a triple change in a bid to get back on level terms and substitute Marc Richards missed a gilt-edged chance to level when he headed past a post with only Bartlett to beat. But Vale eventually equalised when full-back Adam Yates lashed home a low drive from just inside the area with 78 minutes on the clock.
And Vale struck again six minutes later to claim the spoils, Richards scoring from the penalty spot after Benoit Dalibard had brought down Jennison Myrie-Williams in the area.