c++: cross interactions within dual hierarchy

In a C++ application, I have two class hierarchies, one for Workers and for Stuff. I want each sub-class of Worker to interact differently with each sub-class of Stuff through a do_stuff function. I'm not sure how to do that without systematically down-casting both the worker and stuff inside do_stuff and have specific treatments for each possibility.

Moving do_stuff to be a method for either Worker or Stuff and overriding in the sub-classes removes the need for one down-cast, but not the other.

Is there a specific good pratice to handle similar cases ?

Code structure is as below:

class Worker;
class Worker1 : public Worker;
class Worker2 : public Worker;

class Stuff;
class StuffA : public Stuff;
class StuffB : public Stuff;

void do_stuff(Worker worker, Stuff stuff);

int main() {
    vector<Worker*> worker_vec = whatever;
    vector<Stuff*> stuff_vec = whatever;

    for(Worker* worker : worker_vec) {
        for(Stuff* stuff: stuff_vec) {
            do_stuff(*worker, *stuff);
        }
    }
    return 0;
}