What speaks against static fields?

I have two cases in our tool where I have some difficulties in my integration tests with static fields. In the application these fields are no problem. In my tests I create basically a new application with each test method. When I run these in one test run, you can imagine, there will be some problems. First let me show two such cases.

Case 1

class SomeClass
{
    private static IService service;

    public static void Initialize(IService service)
    {
        SomeClass.service = service;
    }

    public void DoSomthing()
    {
        service.Foo();
    }
}

Basically objects of this class will be created in huge numbers. To be able to use the IService object it is stored as an static field.

Case 2

abstract class BaseClass
{
    private static readonly Lazy<SpecificClass> specificClass = new Lazy<SpecificClass>(() => new SpecificClass());

    public static SpecificClass SpecificClass
    {
        get { return specificClass.Value; }
    }
}

class SpecificClass : BaseClass
{

}

This is even more troublesome for my tests, because even if I create a complete new application, when the SpecificClass was used in the first test it is the same object in the second test.

I know, that tests typically show design flaws, but I cannot see one here. The only argument for removing these static fields I can think of is, that my tests do not work otherwise.

So my question now, why is it in these to cases considered bad code to use the static fields or is it not?