The quartet will not be offered new deals as boss Dean Smith looks to re-shape his squad.
Defender Purkiss joined the Saddlers in 2012 and made 85 appearances while left-back Benning came through the ranks at the Banks’s Stadium and leaves following 53 games and two goals.
Striker Grimes arrived on a free from Bury last summer but scored just twice in 36 games while former trainee Heath failed to make an appearance for the club.
The Saddlers are continuing talks with Player of the Season Richard O’Donnell, who is out of contract.
James O’Connor, Paul Downing, Adam Chambers, Craig MacGillivray, James Chambers, Matt Preston, Amadou Bakayoko, Liam Kinsella, Jordan Murphy and Liam Roberts are also holding talks with the club.