The 51-year-old, who has left his position at Wigan, has a wealth of experience having enjoyed a lengthy relationship with Steve Bruce where he was number two at Birmingham, Wigan and Sunderland.
Millers boss Steve Evans has been keen to swell his coaching numbers for some time and Black, who has managed Motherwell and Coventry, joins assistant boss Paul Raynor, fitness coach Paul Warne and goalkeeping coach Andy Dibble in the New York Stadium dugout.
Evans told the club’s official website: “I became aware that there was a possibility Eric might leave Wigan and after making contact, I am delighted having met him that we managed to put something together.
“Eric is fundamentally a coach who arrives here with impeccable references given to me by the likes of Craig Brown, Steve Bruce and Gerard Houllier.
“He has a wealth of experience and knowledge as a coach from a career spanning over 20 years and he will have a major part to play at Rotherham United going forward.”