Visitor pattern knows about subclasses

I've read that a class should not know about its derived types, which seems natural. However, if I want to implement a visitor design pattern, then the base class has to know about it. The visitor itself has to know about the derived classes, so, by transitivity, the base class is aware of derived classes, through the visitor, despite not directly having references to them.

If the base class is part of a library, the visitor should also be part of that library, since the base class depends on it. However, if at some point you need to add new derived classes to extend that library, then you also need to modify the visitor, which is part of the library, which you're not supposed to modify.

So is it possible to have something similar to the visitor pattern but without the base class being aware, even indrectly, of its derived classes ? Thank you.