If we are using Factory Pattern should we hide products from user?

Let say I have an AnimalFactory class which produces Dog, Cat and Fish. Using the AnimalFactory makes things incredibly simpler and everything is fine.

My question is that Is there a rule or good practice that suggests us hide Dog, Cat and Animal from being used directly? I mean Dog myDog = new Dog(); will be forbidden in the code.

If there is, how can we hide them properly?