Sometimes you can clear a key in OS X however it will hold onto it in memory. This is due to OS X holding onto the keys with a time to live forever unless the user logs out or the system is rebooted. You can view the keys in memory by using the following command:

ssh-add -l

You can also clear it manually by using the following command:

ssh-add -D