vendredi 1 novembre 2019

Efficient storage and fast recovery of incremental list changes by timstamp

I have a list-object which can change over time.

For example, at 2001-01-01 10:00 it can be list('a' = 1), then 5 min later at 2001-01-01 10:05 it changes to list('a' = 1, 'b' = 2), then at 2001-01-01 10:10 it becomes list('a' = 3) etc. Sometimes it may not change at all.

The list in reality has many attributes and can contain lists within lists. The contents are primitive objects like strings, integers, ... and lists.

I want to store the changes efficiently, but also make the recovery of the list at a certain point in time computationally efficient. Something similar to how subversion keeps track of file changes, but then for lists.

One obvious way is storing each new list within a parent list, like so:

list('2001-01-01 10:00' = list('a' = 1),
     '2001-01-01 10:05' = list('a' = 1, 'b' = 2),
     '2001-01-01 10:10' = list('a' = 3))

Compressing this with memCompress() will make storage efficient, but recovery rather slow since the whole parent list needs to be decompressed first.

Anyone has any interesting patterns for solving the above?

Aucun commentaire:

Enregistrer un commentaire