Which C++ pattern to use for a library that allows extending its classes?

I am trying to split some code off my C++ simulation software into a library, so it can be used more flexibly. The simulation is based on a Lattice, consisting of a number of Nodes which hold lists of pointers to their Neighbors. Although neighbors are also nodes, I'd like to have a little wrapper class around the *Node pointer in order to implement additional logic/fields (e.g., bool is_neares_neighbor or so.

My class architecture looks therefore something like this:

class Lattice {
private:
    vector<Node> _nodes;
};

class Node {
private:
    vector<Neighbor> _neighbors;
};

class Neighbor {
private:
    Node * _node;
};

So far, so good. Now, I would like my library to handle all the lattice-related logic, but nothing else. However, when using the library in some project, the three classes (Lattice, Node, Neighbor) will carry more logic and fields. The user should therefore be able to inherit from these classes and implement his/her custom stuff, while the library still handles all the necessary lattice-related logic.

What is the recommended way to do that? Are templates appropriate here? In a templated situation, my class hierarchy would look like this:

template<class N>
class Lattice {
private:
    vector<N> _nodes;
};

template<class NN>
class Node {
private:
    vector<NN> _neighbors;
};

template<class N>
class Neighbor {
private:
    N * _node;
};

As you can see, both Node and Neighbor need to know the type of each other, which is a circular condition I have no idea how to deal with here. In addition, the whole library would have to live in header files.

How are situations like these dealt with in the C++ world in the most elegant way?