vendredi 5 juin 2015

Request Aggregator / Middle-tier design pattern for costly requests

I'm working on a program that will have multiple threads requiring information from a web-service that can handle requests such as: "Give me [Var1, Var2, Var3] for [Object1, Object2, ... Object20]"

and the resulting reply will give me a, in this case, 20-node XML (one for each object), each node with 3 sub-nodes (one for each var).

My challenge is that each request made of this web-service costs the organization money and, whether it be for 1 var for 1 object or 20 vars for 20 objects, the cost is the same.

So, that being the case, I'm looking for an architecture that will:

  1. Create a request on each thread as data is required
  2. Have a middle-tier "aggregator" that gets all the requests
  3. Once X number of requests have been aggregated (or a time-limit has reached), the middle-tier performs a single request of the web-service
  4. Middle-tier receives reply from web-service
  5. Middle-tier routes information back to waiting objects

Currently, my thoughts are to use a library such as NetMQ with my middle-tier as a server and each thread as a poller, but I'm getting stuck on the actual implementation and, before going too far down the rabbit-hole, am hoping there's already a design pattern / library out there that does this substantially more efficiently than I'm conceiving of.

Please understand that I'm a noob, and, so, ANY help / guidance would be really greatly appreciated!!

Thanks!!!

Aucun commentaire:

Enregistrer un commentaire