Ba has signed a three-year deal at The Valley after making just three appearances for the Black Cats since arriving in 2013.
The 22-year-old scored on his Sunderland debut in an FA Cup clash against Carlisle but spent last year on loan at Bastia.
“It’s very nice to be a Charlton player,” Ba told www.cafc.co.uk. “I’m very happy to be here. It was a little bit difficult at Sunderland.
“I was young and didn’t have a lot of time on the pitch, but now I am older and ready to fight at Charlton.”