Singleton Vs Singleton Factory

We have a system which has a lot of model objects (e.g. Car, Pedestrian, Road, ...)

Currently all of them have managers (CarManager, PedestrianManager, RoadManager) that return a singleton of the respective class.

An alternative proposed is to have a ManagerFactory singleton that can return instances of CarManager, PedestrianManager, RoadManager. (e.g. ManagerFactory.getInstance().getCarManager())

We also write test for the project and the concern was that if we will use Dependency Injection we will need an actual instance of an object to inject managers.

Is this alternative a good one? Would you change the singleton into something else in this case?