The high-flying Robins completed their sixth straight win on Saturday at Edgar Street, but they had to fend off their struggling hosts in the last 15 minutes of a hard-fought encounter.
Paul Caddis' penalty and Paul Benson's strike looked to have secured the win for Di Canio's side before a spirited fightback from the Bulls, who reduced the deficit with a powerful shot from midfielder Rob Purdie. But the visitors held on for three crucial points.
"This is the type of game you have to win if you are going to win the title" said Di Canio. "I am very happy because of the way we won after being under pressure.
"We were two up and thought it was going to be easy, but once again Hereford proved that they are difficult to beat. We remember the game against them early in the season when we drew 3-3 when they scored in the last minute.
"Once again they showed they have character, quality. But fortunately, for me and my players, we have won the game. We defended well in the last 20 minutes and so I am very happy and proud with the performance."
Any reproduction, publication or redistribution of this material without the written agreement of 365 Media Group is strictly forbidden.