I have a stream of stock prices for 1000 different stocks arriving from the exchange into my code. Due to the very high number of updates I need to use a multithreaded environment to process the price updates.
In a simple world I would have one job queue and a set of worker threads processing the queue. In the real world, however, I can't do this as I need to maintain the time order of updates for individual stocks.
A simple solution would be to use a hashing algorithm that allocated a specific stock to the same thread each time an update was seen. Unfortunately if I do this means that I need to understand the update rates of each of my stocks to ensure my threads are kept busy (update rates are not constant for each stock). If I don't some threads could be saturated whilst others are idle.
Does anyone know of a pattern that caters for this problem?
Aucun commentaire:
Enregistrer un commentaire