Good chain of builder pattern

i have a complex business object like so

public class BusinessObject {

    public Team1Object object1;
    public Team2Object object2;
    public String debug;
    ...

}

I am using slight modification of chain of responsibility pattern to build the above object.

Here is my interface

public interface BusinessObjectAppender {

    public void append(BusinessObjectBuilder businessObject, Context someApplicationContext);

}

Now teams can comes and write their appenders. like so

public class Team1ObjectAppender implements BusinessObjectAppender {

    public void append(BusinessObjectBuilder businessObject, Context someApplicationContext) {
        Team1Object object1 = somehowComputeObject1();
        businessObject.object1(object1)
    }
}

public class Team2Appender implements BusinessObjectAppender {

    public void append(BusinessObjectBuilder businessObject, Context someApplicationContext) {
        Team2Object object2 = somehowComputeObject2();
        businessObject.object2(object2)
    }
}

By using this approach, In case of complex object construction the logic does not bloat up.

But It also has issues like

1. There are no guardrails around Team1 messing up with another team's object or being dependent on another team's data. Apart from code reviews.

2. The cases where BusinessObject is polymorphic, Once i have created builder of 1 type it is not possible to change it in appenders.

Question

1. Is this right pattern to do?

2. what are other ways to achieve the same thing? (creating complex objects in scalable, understandable way)