Let's say I want a system where a service B
's state should be updated according to changes in the state of service A
. When A
's state changes it can either publish
- its entire state
- only its changes in state
So let's say for example that A
's state at points in time looks like
t0 | [ ]
------------------
t1 | [X, Y]
------------------
t2 | [X', Y]
------------------
t3 | [X', Y, Z]
------------------
t4 | [X', Z]
It could publish each of those snapshots of its current state (right column) or it could publish only the changes like
-----------------------
delta(t0, t1)| +X, +Y
-----------------------
delta(t1, t2)| X->X'
-----------------------
delta(t2, t3)| +Z
-----------------------
delta(t3, t4)| -Y
which in theory could create an optimization. But a few things I see that would have to be guaranteed are
- Published events are consumed in correct order
- Published events that were "missed" by a service need to be recognized and somehow compensated for
- There is no bug in the delta algorithm
whereas if the entire state is published then
- If 2 published events are consumed in the wrong order then publishing a 3rd event corrects everything
- If a published event fails to be consumed then publishing a 3rd corrects everything
- No delta algorithm, so less complexity and compute resource on the publishing service
A few solutions I can think of to make the delta approach a little more robust add a lot of complexity:
- Publish both the changes and the entire state and a pointer to the previous event. The services consuming the event can figure out whether to process the delta or the whole data set.
- "Cycle" between publishing deltas and entire state. Maybe publish changes 9/10 times and then the entire state the other 1/10 of the time just to help correct things.
Both of those feel weird. I am interested in whether there exists a robust solution for notifying of "additions, removals and modifications" while ensuring eventual consistency in the system.
Aucun commentaire:
Enregistrer un commentaire