The hosts would have climbed into the top three with two games left had they beaten their promotion rivals.
But ex-Bury midfielder Worrall, whose son recently passed away after being born prematurely, came off the bench to score a 74th-minute free-kick.
Southend then hung on for a sixth straight win to go level on points with third-placed Wycombe.
“You’ve got to take your hat off to David Worrall,” said Brown. “He’s been through the mill in the last few weeks. He wants to dedicate that goal to his son, who was born premature and died after 45 minutes.
“We had to keep it under wraps but that’s the main reason he’s not been playing. It was a horrible situation for him but the whole changing room was behind him and you could see that by the celebration.
“He doesn’t take free-kicks in training but he demanded that ball and stuck it in the top corner.
“Bury were by far the better team in the first half so I had to get into our players at half-time to make them realise how important this game was.
“The response in the second half was there for everybody to see. The character of the team was tested and, my word, did they stand up to it.”
Having made the match free entry, Bury were cheered on by their biggest home crowd in almost 18 years and bossed the first half.
Andrew Tutte and Nathan Cameron went close early on before Danny Mayor’s shot was deflected over.
Tom Soares then headed wide while Joe Riley curled a free-kick wide after the break.
Southend then got a foothold in the game and Gary Deegan’s long-range strike stung the palms of Bury keeper Nick Pope.
And although Bury sub Danny Rose drew a straightforward save from Daniel Bentley, the Shrimpers took the lead moments later.
Kelvin Etuhu fouled Jack Payne 20 yards out and sub Worrall curled it over the wall and into the top corner.
Bury pushed for an equaliser but Cameron headed wide from a Craig Jones cross before Bentley made a point-blank save to deny Ryan Lowe at the death.
Bury boss David Fitcroft said: “I’m disappointed with the result, our first-half performance was as good as anything we’ve done this season, but we didn’t work their keeper enough.
“The fans have turned out in force and the players gave them as much as they could, without quite finishing the work off.
“The second half was more even. It wasn’t a free-kick but don’t take anything away from David Worrall’s strike. It was on a postage stamp, a fantastic finish which deserved to win any game.
“The players are hurting, they’re frustrated, but this isn’t over.
“We’ve got two more massive games for the club to try to achieve six points and go up. Stranger things have happened.
“They’re a good group, they’ll get round each other and drive each other forward.”