We have a SQL table which the primary key is a guid. We want to periodically query some service - and unfortunately these have to be individual queries per row - and update all of the rows with new values.
We have a distributed system which we can run separate update queries on separate nodes concurrently.
My current design involves around retrieving N rows at a time from different nodes, so the work is splitted. Perhaps add another index field which will help this splitting mathematically (like an int). Another potential design would be to queue all of the update queries and use auto-scaling to handle them as quickly as it can.
Just wanted to see what other types of architecture can be planned around a requirement like this.
Aucun commentaire:
Enregistrer un commentaire