Striker Lowe nodded in unmarked in the final minute of the first period to move the visitors to within two points of third-placed Wycombe, having played a game less, and secure a club-record seventh successive away win.
Pompey have lost both their games under caretaker boss Gary Waddock after Andy Awford was sacked on Monday.
Jed Wallace forced Nick Pope into two saves in a minute as the midfielder caused problems with his free role behind the strikers.
Wallace was at it again when he latched on to a Craig Westcarr dinked pass but Pope was equal to it, and it was the 21-year-old who jinked past Adam El-Abd before dragging wide in the 37th minute.
Despite the hosts’ dominance it was Bury who took the lead on the cusp of half time. Joe Riley delivered a pinpoint cross after a patient move and substitute Lowe nodded in his 10th of th eseason.
Lowe nearly added a second in the 46th minute but his 25-yard free-kick flew over.
It did not matter, though, as Portsmouth failed to mount a comeback and suffered a third straight defeat.