New .net core project: Every class registered as a singleton

I've just inherited some code, one thing that set alarm bells ringing in my head was that every class is registered as a singleton, and each singleton is injected into the other singletons as if you were using DI with instance methods. Example:

// In startup
services.AddSingleton<ISingletonClass, SingletonClass>();
services.AddSingleton<IAnotherSingleton, AnotherSingleton>();

// In AnotherSingleton class
private readonly ISingletonClass _singletonClass;

public AnotherSingleton(ISingletonClass singletonClass)
{
    _singletonClass = singletonClass;
}

Now the guy I replaced has a ton of experience with an MSCA (Microsoft Certified Solutions Associate) and some of the code he's written is actually quite nice. He also kept a 70% test coverage and good tests, not just tests for tests sake.

1. What would the reason for singletons everywhere?
2. I know one of the main requirements of the system was performance. Is there a performance gain by accessing an already instantiated class rather than instantiating each time?
3. Any other reason for coding this way?
4. As I'm going to be required to take over this code base, should I change this implementation sooner rather than later as it may cause me headaches down the road? What would those possible headaches be?