How to avoid infinite loop on swift property observer

I’m trying to implement a logic that resembles the functionality of a “Group” layer in some popular design apps.

Let’s say I have a class in my domain that can have some children and a parent of it’s same type. I want that when I update the frame of the parent, the children adjust their frame proportionally. Likewise, when a child's frame change, I’d like the parents frame to get updated wrapping its children accordingly.

class Layer {
    var parent: Layer?
    var children: [Layer]

    var frame: CGRect {
        didSet {
            if parent != nil {
                // Tell parent to update its frame
            }

            children.forEach { (child) in
                // tell the child to update it's frame
            }
        }
    }
}

When the parent’s frame gets updated, he tells his children to update their frames, then the children get back to the parent asking to update it’s frame. And we get caught in an infinite loop.

I was wondering which would be the correct way to implement this and if there is any design pattern that addresses this case.

Thanks.