The on-loan defender from Swansea produced a perfect finish, hooking in a right-wing corner from Chris Burke in the 92nd minute.
Two goals from Scott Allan had earlier appeared to have put Birmingham into the second round, however Lewis Alessandra hit back by netting twice for the visitors to keep battling Plymouth in the game.
A low key and goalless first half, when neither side created many scoring opportunities, contrasted sharply to the second period.
Plymouth’s best chance of taking the lead in the opening half was a header from Marvin Morgan which produced a high-quality save from Colin Doyle.
Undoubtedly the miss of the game came two minutes into the second half with Plymouth wrong-footed.
Lee Novak squared his pass to Burke who amazingly scooped the ball over the bar in front of an open goal.
Fortunately for Birmingham newcomer Allan, on loan from West Brom, was on the mark in the 49th minute when he took on a pass from Tom Adeyemi to score with a low cross shot.
Birmingham should have added to their lead two minutes later when Andre Blackman brought down Burke to concede a penalty.
Novak took the spot kick only to see Jake Cole dive to his right to make a superb save. Cole later thwarted Novak again with a reflex goalline save.
The Birmingham misses were followed by Alessandra equalising for the first time with a goal out of the blue when he scored with a speculative 25-yard drive into the top left-hand corner.
Allan was on hand in the 83rd minute to notch Birmingham’s second with a powerful first-time shot to convert a cross from Shane Ferguson.
Yet in the second minute of stoppage time Alessandra popped up again to notch his second goal.
Birmingham might have feared then it would not be their night, but any doubts were eased in the second minute of extra-time thanks to Bartley.