Circular dependency involving Observer design pattern

I am working on a multi-layered project that involves a server and some clients.

Most of the messages are initiated by the client and sent to the server, meaning they look like this: Client -> Intermediate Layers -> Server

But some messages are server-initiated updates, meaning they're reversed. So I need to implement the Observer pattern, where the client is the Observer and the server is the Subject. The problem is that the client already has a dependency on the server. This means that I can't let the server have a reference to the client, because then I would have circular dependency.

How do I solve this issue? One idea that I've read about is using a mediator like this: Server -> Mediator Client -> Mediator

But I've failed to understand how to make it work.