Can I use factory pattern to instantiate objects, with data from an external API?

I'm making a website, through .NET Core 2.2, which is basically just showing different kinds of maps, which I receive as IIIF images.

The images are coming from an external REST API, and I need to make the application scalable, testable, and open for integration with other apps.

It's a proof of concept, so at first, I will just make an endpoint, which can serve data to a basic web MVC frontend.

If possible, I want to be using Factory Pattern, for creating my map objects. I don't know which variation I need to be using if it's possible.

My plan as of right now is to represent the JSON objects from the API, in repositories. Then I want a factory to decide, which type of map should be created. But I'm REALLY confused as to how I should approach this.

This is where i have gotten stuck:

public class MapManager 
{
    AbstractFactory mapFactory = new MapFactory();
    IMap cadastralMap = mapFactory.CreateMap("Cadastral");
}

public abstract class AbstractFactory 
{
    CreateMap(string title);
}

public class MapFactory : AbstractFactory 
{
    public IMap CreateMap(string title) 
    {
        if(title == "Cadastral") 
        {
             return new CadastralMap();
        } 
        else
        {
            return new NauticalMap();
        }
    }
}

public interface IMap 
{
    string Title { get; set; }
}

public class CadastralMap : IMap 
{
    string Title { get; set; }
}

public class NauticalMap : IMap 
{
    string Title { get; set; }
}    

The MapManager is basically just a controller, but I named it manager because the API already had classes named controllers. The idea is that the API-controllers will call the managers.

Right now I'm confused as if this is the way to go, and how I would include the response from the API in this.

Is it possible, and is it the way to go for me? Any help on how to do this, or what my backend architecture should look like, is very much appreciated. A small extra thought, maybe I should be using dependency injection, and inject the factory into the constructor of MapManager, but as I said, I'm quite confused.