Are there any downsides of invokable factories?

Are there any downsides of invokable factories? For example:

class EntityFactory
{
    protected $map = [
        'User' => User::class,
    ];        

    public function __invoke(string $entity, ...$args)
    {
        if (! $class = $this->map[$entity] ?? null) {
            throw new InvalidArgumentException(sprintf('Unknown entity "%s"', $entity));
        }

        return new $class(...$args);
    }
}

And later:

$entityFactory = new EntityFactory;
$john = $entityFactory('User', 'John');
$jane = $entityFactory('User', 'Jane');

Am I missing something? Thanks!