What happens to the extrinsic state when using the Flyweight Design Pattern?

I am currently researching software design patterns and have been somewhat perplexed with the flyweight design pattern's implementation.

As far as I understand, the pattern works by separating the shared/constant intrinsic data from the unique, changing extrinsic data. The intrinsic data is kept in a set of objects that are managed by a factory class to prevent duplicate intrinsic data objects. This I understand.

What I don't get is what I do with the extrinsic data. In the examples I've seen which have mostly been about drawing shapes for some reason, the extrinsic data is just brought up to a higher level where it is passed into a method in the intrinsic object. Is this always the case or is this just a result of everyone using the shapes example?

One thought I had is to use composition to add a reference to the intrinsic object into an object with the extrinsic data. The only additional space in the object is the size of the reference. Would this work as a good example of the flyweight design pattern?

Thanks!