How do I cancel my emails being forwarded to an address I no longer hold?

I have done exactly as instructed: removed that old address from the email forwarding section and replaced it with the new address I want them forwarded to. Despite it clearly saying you can only forward to one address, they are now going both to the old address and the new one! Unfortunately that old address goes with the job I have moved from so my successor is getting to read all my private emails! This has now been going on for over a month despite multiple attempts to stop it.