Should you use the AtomicReference for the Singleton Pattern?

I ran across the AtomicReference class and was wondering if this could be a good way of creating a Singleton that is mutable and could be replaced for testing.

I know the double-lock checking has issues so I didn't want to go that route. Also I would prefer to lazily instantiate the singleton instead of initializing it. For example, during testing I wouldn't want to use the default class SimpleExample but in certain cases it is either expensive to create or problematic in certain environments. I would prefer to be able to replace it prior to the request.

public class SingletonExample {

  private static AtomicReference<SingletonExample> sInstance = new AtomicReference<>();

  private SingletonExample() {
  }

  public static SingletonExample getInstance() {
      return sInstance.updateAndGet(u -> u != null ? u : new SingletonExample());
  }

  public static void setInstance(SingletonExample singletonExample) {
      sInstance.set(singletonExample);
  }
}

Here are my questions:

1. Is there a big performance hit?
2. Is there an issue that I am not aware of using it as a Singleton?
3. What are reasonable alternatives?