mardi 31 mai 2016

What is the best way to calculate values over time from incoming stream

Running a C# .net app that receives data every 30 seconds from 100 clients and then stores data in a database. The data is for two parameters for each client. I need to determine the total for each parameter, for each client per hour and make decisions based on the results. The decisions algorithm would be making decisions for the last hour worth of data in a sliding window fashion. My initial thinking is to keep a dictionary of those 100 clients with key being client IP, and value being a running total. However 1) if my app restarts half way through the hour or at minute 59, I lose all those warm running totals. 2) if more clients start sending data, dictionary will consume more memory, 3) if in the future the 2 parameters become 100, the dictionary grows even bigger 4) making the running total value always reflect one-hour worth of recent data is not straightforward.

Is there any different approaches I should consider? best-practice? design patterns?

Thanks.

Aucun commentaire:

Enregistrer un commentaire