Understanding Factory Pattern with its uses

I have the basic working knowledge on C# and not have the intention of extending it to patterns and design strategies. I came across "Factory" pattern and have few clarifications. Here it goes :

public class Factory
    {
        public string MyProperty { get; private set; }

        private Factory()
        {
        }

        private Factory(string myProperty)
        {
          MyProperty = myProperty;
        }

        public static Factory CreateObjectbyCallingConstructor(string myProperty)
        {
            return new Factory(myProperty);
        }

        public void Display()
        {
            Console.WriteLine("Hello" + MyProperty);
        }
    }

Here's how I invoke it :

 static void Main(string[] args)
        {
            Factory f = Factory.CreateObjectbyCallingConstructor("there");
            f.Display();
            Console.ReadLine();
        }

I understand the following concepts:

a) It is not possible to create instance because of private constructor.

b) I have a static method which is going to create an instance along with parameterised constructor. So I get an instance .

Now, the same thing can be achieved using a normal class instantiation without having private constructor. My question what business problem it solves for me to go for Factory pattern ?

Can anybody explain in layman's language so that I could relate it easily also with immutability with comes with it?