Pattern for a list that can contain two types of items

I'm trying to figure out a nice way to adopt an API change we've recently undergone that "weakens" a type on one of our objects.

Users used to be able to provide a list of references to "Dogs". We've made a decision that we now accept "Dogs" and "Cats".

The client library uses strongly typed references, so the property currently looks like:

public List<IReference<Dog>> occupants { get; set; }

I considered changing it to:

public List<IReference<Animal>> occupants { get; set; }

But I don't like that what used to be compile-time validation is now run-time (and server-side) validation. I would have preferred to have something more strongly typed like:

public List<Occupant> occupants { get; set; }

public class Occupant : IReference<Animal> {
    IReference<Animal> _internalReference;
    public Occupant(IReference<Dog> dog) { _internalReference = dog }
    public Occupant(IReference<Cat> cat) { _internalReference = cat }
    //... (Implement IReference<Animal> as a pass-through to _internalReference)
}

Unfortunately, because C# does not allow implicit operators to convert from an interface, existing users updating their client library would have to undergo hundreds of lines of code changes to "wrap" all their existing Dog references in an Occupant constructor.

So next I considered creating a new class for the list with some helper methods:

public class OccupantList : List<Occupant> { 
    public void Add(IReference<Dog> newDog) => base.Add(new Occupant(newDog));
    public void Add(IReference<Cat> newCat) => base.Add(new Occupant(newCat));
    // For backwards compatibility with new list assignments
    public static implicit operator OccupantList(List<IReference<Dog>> legacy) =>
        new OccupantList(legacy.Select(dog => new Occupant(dog)));
}

Unfortunately, C# exposes no way to override the core ICollection.Add implementation behind the scenes, which is used by all sorts of reflection utilities and my JSON serializer to populate this list, so it complains when it's given objects to add that aren't already of the type Occupant. I personally encountered issues with the JSON deserializer I was using. I could write a custom deserializer for this class but I took it as a sign that trying to inherit List<Occupant> and add support for types that don't inherit from Occupant was a bad idea doomed for failure.

Does anyone have any ideas or examples that can might steer us in the right direction?