It’s been almost 2 years since my last trip but unless you are driving into a township, I wouldn’t be worried. I’m planning a trip in 2026 and unless something really changes I have no concerns.
Gardens, Sea Point and Camps bay are all areas I would be surprised if you weren’t comfortable walking around.
Compared to NYC, Chicago or Houston I think Cape Town is equally safe from the standpoint of - if you aren’t in bad areas or doing “dumb” things you should be perfectly safe.
Happy to answer more questions via PM as well.