I'm writing a node application that pops messages onto a Kue job queue from REST calls processed by Express. I then want a number of worker processes running across multiple threads and servers to consume these messages and perform their long running tasks.
I have successfully written a standalone app that seems to work OK, but I'm confused about how I can distribute this across multiple servers. I've been looking into PM2 which seems to be the best way to load balance node processes.
My questions are..
- Would it be best that express/Kue message producer app be a separate app to the workers apps? So for instance, one Express app running on one server, then a worker app running separately on all the servers?
- Can I stop/start all/some of the workers across different servers with a single instance of PM2? Or, do I need an instance of PM2 running on each server?
- Can anyone offer up a better solution?
The worker processes are basically copying/moving large files to and from remote servers, hence the need for distribution across multiple servers instead of just threads, for disk i/o and bandwidth performance.
You input is much appreciated.
Aucun commentaire:
Enregistrer un commentaire