Argyle central defender Maxine Blanchard scored an 84th-minute equaliser having seen his 18th-minute header come back off the bar, four minutes after Rovers' top scorer Matt Harrold headed his 13th goal of the season.
McGhee said: "I'm disappointed. I never really thought we played or got hold of the game. We possibly shaded the first half, but the difference was that we had Mustapha Carayol, who was taking the ball to them and making chance after chance that we weren't able to take.
"Regardless of how well or badly we played, we had chances to have wrapped the game up - and we didn't take them. Ultimately, that is what has cost us.
"Muzzy (Carayol) probably had the best chance for a second goal but got a bit flash when all he had to do was put it in the net. He did brilliantly at times and scored a fantastic goal on Saturday. But he had an easier chance here and missed it.
"There were others who had chances too. Matt Harrold should have had a second goal with another header. I'd like to think he'll be going home not thinking about the goal he scored but rather the one he missed. There were opportunities not just to score but to make other chances - but we made bad decisions in certain situations."
Any reproduction, publication or redistribution of this material without the written agreement of 365 Media Group is strictly forbidden.