Why use factory design pattern over constructors and application logic?

I have come across the factory pattern many times while I have been coding and struggle to understand when and why you would use it over constructors, application logic. I believe that understand how the factory pattern works and what the benefits of the pattern are said to be (please do correct me if I am wrong or missing something), however I feel the benefits have associated issues that discourage the use of the pattern. I figured either there must be gaps in my knowledge regarding the pattern or that I have misunderstood how it should be used and would be very grateful to anyone who could point me in the right direction.

My understanding of how it works:

• You have an abstract class that works like a template (e.g. IBankAccount).
• You have several subclasses that inherit the abstract “template” class (e.g. CurrentAccount, SavingsAccount).
• You have an interface that defines the factory methods (e.g. Create, Read, Update, Delete) which return / expect the abstract “template” class, meaning that any derived class of the “template” can be used.
• You have a factory class which inherits the factory interface where the definitions have logic applied to them.

Core benefits I have heard, found, researched:

1. Ability to handle multiple types. By referring to the abstract “template” to can pass and return multiple derived types (e.g. CurrentAccount, SavingsAccount)
2. Promotes decoupling. It removes dependencies meaning changes to the code can be made with minimal impact. (e.g. by having a factory to do Create, Read, Update and Delete, there is less need for them to be changed and classes do not depend on them or other objects.
3. Unit testing. The factory pattern uses interfaces of which it is the nature to provide many implementations, thus enabling mocking. By using interfaces you’re not creating dependent types that can be broken and therefore break your tests as well.
4. Design pattern. As with all design patterns they are meant to be a standardized approach to a common problem to make it easy for programmers to recognize and work with.

Thoughts & issues regarding each benefit:

1. Generic type vs Exact type. While returning multiple types is useful, this pattern will return the abstract “template” class so any properties specific to a derived class will not be accessible. Why would you choose to do this instead of a switch statement in your application that could return the exact type with all its related definitions? I understand you when using the pattern, you could cast it back to the correct type but retrieving derived specific properties at this point requires further expenditure (potentially calls to a database) which would have otherwise been done on initialization.
2. Decoupling without a factory. A lot of the decoupling benefits are achieved through the object class inheritance and not through the factory itself. If you have common methods in the base class, such as create, then this code is less likely to need to be changed and does not make assumptions of the derived classes. I do acknowledge that interfaces while coding do help to enforce structure, also that decoupling generally is so important, however this feels more like decoupling for the sake of decoupling.
3. Alternative unit testing solutions. I have no argument regarding the benefit of unit testing, it does support and encourage it, but it should be noted that with todays mocking frameworks and tools there is very little if any overhead by not using the factory pattern.
4. Design pattern overkill. Having design patterns to encourage a common approach to problems is undeniably useful, however my thoughts above lead me to ask the question at what expense? Because I struggle to see a viable reason for it, it seems to me like a lot more unnecessary work.