Metered taxis are controlled by the city council and can be considered safe and reliable. The price per kilometer is around R8-R10 and can often be read at the taxis side door. You can also set a fixed price with the driver, especially when going to a far away destination.
Minibus taxis are only advisable for locals. They cover most of the Cape Town Metropolitan Area and are very cheap. But they are crowded and less safe than normal taxis.
There is a network of public buses (Golden Arrow buses) that offer cheap and fast connections for example from the city centre to the Waterfront. They run mostly during the day.
The best way to get around in the City Centre cheaply is to hire a Rikki. This multi-passenger mini-minibus zips around from place to place. No guarantee you'll get to your destination directly, but it is safe, fun, and all the drivers are characters.
You can also rent a Vespa or other brands of scooter. A number of places in the city offer this service and it is a great way to be mobile and save over renting a car