Real time message push to embedded devices

I'm designing firmware for the first time, and I've come across a pattern that I'm not sure how to solve. I need to be able to push arbitrary messages to embedded devices. The contents can be anything from octet stream to json. The embedded devices are not directly accessible over the network, however they can call out to the internet.

The obvious solution would be polling. But I'd like to use a push mechanism for more real time communication if possible.

Is there a scaleable way, for example, to open a socket from the client side (embedded device), and then hold it open to allow message pushes from the server side (cloud)?