useFactory depending on current component

I want to implement an Abstract Factory pattern. Here is a factory:

   export abstract class MyFactory {
      abstract getAdapter(): DataAdapter;
    }

Lets say we have 2 components AComp and BComp which belong to the same module AModule. Both of them have the factory dependency:

  constructor(
    ...
    private factory: MyFactory
  ) {}

I wan to hide concrete factory creation logic. So the solution I came up with is to put the creation logic into AModule:

@NgModule({
      declarations: [AComp, BComp],
      providers: [
        {
          provide: MyFactory,
          deps: [MY_FACTORY_TOKEN],
          useFactory: (token: string) => {
            // return concrete factory depending on the token
          }
        }
      ]
})
export class AModule { }

And each component has its own token provider:

@Component({
  ...
  providers: [ {provide: MY_FACTORY_TOKEN, useValue: 'string val' } ]
})

But it says No provider for InjectionToken myfactorytoken!. Apparently I can't get the component provider on the module level.

Basically, my question is how to return concrete factory depending on current component? E.g. if it is AComp then return AConcreteFactory, if it is BComp then return BConcreteFactory.