The sides exchanged goals in a chaotic opening five minutes in Temuco, Neymar’s header (5) answering an opener that Brazil’s goalkeeper Jefferson gifted Christian Cueva (3), but it took until the 92nd minute for Costa to prod Brazil to three points.
The result – an 11th straight win – leaves Brazil on three points in Group C alongside Venezuela, shock winners over Colombia in the day’s early game.
The Colombians next face Brazil on Wednesday but – Neymar aside – there was little to frighten them on the night ahead of a game they cannot now afford to lose.
Peru took barely two minutes to give Dunga’s side a bloody nose: David Luiz’s casual pass did not make life easy for Jefferson, but the ‘keeper should still have avoided side-footing straight to Cueva, who banged the ball back past him.
The lead did not last long, Neymar left in all sorts of space to head in a right-wing cross, and the captain would have had a 45th international goal but for a brilliant Carlos Zambrano block.
Jefferson made a meal of a couple of Peru potshots, but it was Brazil who had the better of the scarce chances after the break as Neymar clattered the crossbar from distance and Costa dinked wide at the end of a quick counter-attack.
A draw looked nailed on when Neymar then bent a shot wide on his left foot, but the Barcelona man picked out a pass of pure class to give Costa a second opportunity. This time, he did not miss.