I told my ex I can't be his friend anymore - now he thinks I'm angry?

Don't call him. You did the right thing. You have to heal and you can't if you keep rubbing salt in a wound.