Ex-Fulham trainee Keanu Marsh-Brown set Oldham on course for three points but Wordsworth denied them with his 10th goal of the season.
Colchester striker Freddie Sears blasted an early shot over the bar and Ian Henderson should have done better from Steven Gillespie's far-post cross.
Oldham missed a clear-cut chance after 34 minutes as Marsh-Brown latched on to Magnus Okuonghae's poor header and fired off target when clean through on goal.
Marsh-Brown made amends three minutes later, forcing home from two yards after Jean Yves Mvoto's header was palmed out by keeper Ben Williams. Shefki Kuqi could have doubled the Latics' lead before Colchester sub Kayode Odejayi headed over wastefully from close range.
The U's, however, levelled nine minutes from time as Martin Rowlands sent in a left-wing free-kick and Wordsworth converted a simple six-yard header.
Any reproduction, publication or redistribution of this material without the written agreement of 365 Media Group is strictly forbidden.