Custom erase for c++ set or other collection

I want to do something like this:

std::set<my_type> s;

s.insert(new my_type(...));
...
s.erase(...);

Where the set's erase will delete the pointer to avoid a memory leak.

Is this possible with C++ containers or would the proper solution be to subclass the container and write my own erase, or use some sort of smart pointer scheme?