How to pass the base class on the caller side and accept derived

I have a service that returns the base class response. For sure it has derived responses.

    public class ResponseBase
    {
        public string Response { get; set; }
    }

    public class DerivedResponse : ResponseBase
    {
        public string AdditionalResponse { get; set; }
    }

Here is the service code:

public class SomeService
{
    public ResponseBase GetResponse() => new DerivedResponse();
}

Then I need to handle different responses in different ways. Obviously try to get appropriate behavior via 100500 if\elses is not a good solution. And I decide to have One special response handler for each concrete response.

    public interface IHandlerBase<T>
        where T : ResponseBase
    {
        void Handle(T response);
    }

    public class DerivedResponseHandler : IHandlerBase<DerivedResponse>
    {
        public void Handle(DerivedResponse response)
        {

        }
    }

Also, we need to encapsulate behavior which will decide what handler to get in order to handle the concrete response and I think not a bad solution for that will be factory. And I got a problem with that because I don't know what to return since I don't know compile-time derived type:

    public class HandlerFactory {
        public IHandlerBase<> /*WHAT RETURN HERE*/ CreateHandler(ResponseBase response)
        {
            //decide the derived type of the 'response' and return appropriate handler
        }
    }

For sure we can remove generic parameter, accept base class in all handlers, then convert to special in the concrete handlers, but I don't think it is a good solution. So could you please advise how to do that in a clean way, maybe some patterns or solutions?