Refactoring existing pattern

I am using Factory Pattern in order to initialize objects:

public class Factory
{
    @Autowired
    private HandlerOne handlerOne;
    @Autowired
    private HandlerTwo handlerTwo;

    public <T extends Handler> T createHandler(Class<T> type) {
        switch(type.getSimpleName()){
            case "HandlerOne":
                return type.cast(handlerOne.handle());
    }
}

And this is how I use the factory itself:

handlerFactory.createHandler(HandlerOne.class);

All the Handlers are located in a separate package called com.mydomain.project.Handlers

As you can see, if there are X handler implementations, I'll need to create X variables inside the factory and I think this is bad. Is there an existing pattern I can apply in order to make this process easier?