Ellis powered a 75th-minute header past Northampton keeper Chris Weale from Kevin Nicholson's corner from the right to earn the Gulls all three points.
It was tough on the Cobblers' Kevin Langmead, who made a great diving block to deny Chris Robertson from point-blank range moments before the goal as United upped the ante.
Weale also made a brilliant diving save to palm away striker Rene Howe's 18-yard strike after he spun Langmead and cut inside before letting fly, but Ellis popped up to settle matters.
Earlier, Luke Guttridge had come within a whisker of opening the scoring for Northampton when he dived in to try to convert Michael Jacobs' whipped cross from the left on 65 minutes.
Guttridge had a 15th-minute 20-yard shot saved by home keeper Bobby Olejnik, who also made smart first-half saves from Saido Berahino and Jacobs, but the visitors could not find the goal they needed and Aidy Boothroyd's side have now lost three games in a row.
Any reproduction, publication or redistribution of this material without the written agreement of 365 Media Group is strictly forbidden.