dimanche 31 janvier 2021

Design suggestion for RabbitmQ Producer & Consumer

I have a RabbitMQ design question. Let say I have 6 exchanges and 10 queues and split up as below

  • 5 exchange with type 'fanout' and has 5 queues bound to it
  • 1 exchange with type 'topic' and it is routed to 5 different queues based on the routing key.

I have a microservice application which runs Kubernetes with the scale of 25 and all these applications acquire 1 rabbitmq connection per process. So 25 rabbitmq connections act as producer.

I have another application which also runs in Kubernetes with the scale of 1 and these applications acquire 1 rabbitmq connection. So 1 rabbitmq connection act as a consumer.

Numbers: Let say every exchange gets 100k messages per day.

Tech stack: Node.js + amqplib

Questions:

  1. How many channels should producer needs to create for publishing the messages to exchanges?
  2. How many channels should consumer needs to create for consuming the messages from the queues?
  3. Is it a good approach to have one application act as a consumer which consumes the message from all the queues?
  4. How can I scale the consumers automatically based on the queue size in Kubernetes?
  5. Is it possible to build priority based on consumers? Let say due to heavy load conditions, I would like the consumers to stop consuming from a couple of queues and focus all the resources on the rest of the queues.
  6. How many connections should producer & consumer create?

Thanks in advance :)

Aucun commentaire:

Enregistrer un commentaire