Visitor pattern

When i read about visitor pattern it says like

Allows for one or more operation to be applied to a set of objects at run-time, decoupling the operations from the object structure.

If my assumption is correct, we will define an abstract visitor which holds methods for treating each Objects. Then concrete visitor will implement each of these methods. By this we are separating the logic of handling objects from Object class to visitor implementation.

My doubt is if we have only one visitor implementation, do we really need to use this pattern? can't we just place the implementation in each Object class and call it directly? Please correct me if i miss something in between.