Holdsworth arrived unmarked at the back post to head home substitute Joe Skarz’s cross and avenge the Terriers’ 4-0 mauling at Elland Road in December.
Both sides were reduced to 10 men in a first-half devoid of real goal-mouth action.
Leeds’ captain Jonathan Douglas was sent off for two bookable offences after 38 minutes, while Jon Worthington was dismissed after an altercation with Lubomir Michalik.
After the break Town’s Michael Collins rattled the crossbar with a shot from 20 yards out and Malvin Kamara forced a good stop from Leeds goalkeeper Casper Ankergren.
In added time, the Whites almost snatched an equaliser through on-loan striker Dougie Freedman but he missed from close range.