Pattern "allocate memory or use existing data"

I have field
std::map<std::string, std::map<unsigned int, float>> widths;

I copy data to widths[key] from another map or insert custom data to widths[key] depending on runtime criteria. However, copying is too slow. I am going to use pointer to std::map<unsigned int, float> as widths::value. Thus(depending on runtime criteria):

1. Pointer has address of existing map.
2. Allocate memory and write custom data.

Also I need flags to decide if delete widths::value in destructor. Do you know better pattern to resolve this task?