Upson completes a three-match ban following his recent sending-off against Wigan and Williams remains sidelined by the foot injury he suffered in the same match.
Shaun Cummings is struggling with a groin injury and fellow full-back Carlos Edwards is a long-term absentee as he recovers from cruciate ligament damage.
Striker Lee Gregory could return to the starting line-up after dropping to the bench for the midweek defeat at Blackburn which nudged the lowly Lions ever closer to relegation.
Derby hope to have leading scorer Chris Martin back in contention as they look to seal a play-off place.
Martin has missed the last two matches due to a recurrence of the hamstring problem that saw him sit out 10 games during February and March. The 20-goal striker will continue to be monitored but Craig Bryson and Zak Whitbread are definite absentees.
Midfielder Bryson and defender Whitbread limped off during the first half of the 4-4 draw against Huddersfield with thigh and hamstring problems respectively and both players look like missing Derby’s remaining two Championship fixtures.
Will Hughes (ankle) was injured in the warm-up last weekend and should be fit for Saturday but fellow midfielders George Thorne (thigh) and John Eustace (knee) and defender Jake Buxton (back) remain out.