lundi 28 août 2017

What algo/design pattern do i need to keep limited time series data constantly updated?

I'm a hobbyist coder; never worked professionally. I am not looking for anyone to write code for me, but I need to know how to approach this problem and, perhaps, ideas for further research. This problem is an outgrowth of my kid's science project that I wanted to toy with.

Bottom line is: I have a sensor that feeds data objects at irregular and unpredictable intervals (temp, air pressure, other things). Each new "update" feeds another data object with all the pertinent info...and each object has a time stamp. The unpredictability is the core of the problem, and that will not change. As my code gathers new data objects at irregular intervals, I need to run simple arithmetic operations on the data objects for the last n minutes. I need to output these arithmetic operations continuously on each new "update" of fresh data. The sensor could put out as many as, say, 20 updates per second...or fewer. CPU and RAM usage are a mild concern, but I want to focus on getting working design first.

At first, I was thinking circular array or queue, but that's no good since I do not know how many objects will cover the last n minutes.

Next, I considered a doubly linked list. The problem with this approach is that it will have to consume heavy CPU by iterating over the entire list each new "update" in order to remove outdated objects from the list or will have to consume a lot of ram if I do not remove old objects from the list each time.

I am wondering what design patterns (and data structures) might fit this problem, and what other items I could research to learn more to solve this problem.

I realize I am not giving much information here, but I want to keep simple and I believe I have given the gist of the problem.

I really appreciate any help. Btw, I'm using C# and the CLR for now. Python might be a better option since this is data-science'ish. I believe I could write and/or consume a Python library/class. I'm not real fluent with Python.

Aucun commentaire:

Enregistrer un commentaire