What are the drawbacks of Singleton Design Pattern?

What are the drawback of Singleton Design pattern? Is it memory allocation or anything else??