Rovers striker Matt Harrold sidefooted home a 30th-minute penalty after being brought down by Onismor Bhasera, and the forward doubled the advantage two minutes later with a fine shot on the turn after Joe Anyinsah’s break and cross.
But Warren Feeney reduced the deficit for Plymouth on 52 minutes, pouncing to net from close range after Nick Chadwick flicked a corner towards goal, and Chadwick levelled on 79 minutes by slotting a low left-footed shot past Scott Bevan.
Hemmings ran unchallenged through the middle to shoot the winner with seconds remaining to cue a chorus of displeasure from Rovers fans towards their manager.
Paul Buckle’s men could have been three goals to the good at the interval but for a brilliant diving save by Jake Cole to palm Andy Dorman’s 27th-minute header onto the crossbar.
But Argyle dominated the second half and would have scored more but for two brilliant Bevan saves.