jeudi 11 octobre 2018

Microservices and Publishing State versus State Changes

Let's say I want a system where a service B's state should be updated according to changes in the state of service A. When A's state changes it can either publish

  • its entire state
  • only its changes in state

So let's say for example that A's state at points in time looks like

 t0  | [ ]
------------------
 t1  | [X,  Y]
------------------
 t2  | [X', Y]
------------------
 t3  | [X', Y, Z]
------------------
 t4  | [X',    Z]

It could publish each of those snapshots of its current state (right column) or it could publish only the changes like

-----------------------
 delta(t0, t1)| +X, +Y
-----------------------
 delta(t1, t2)| X->X'
-----------------------
 delta(t2, t3)| +Z
-----------------------
 delta(t3, t4)| -Y

which in theory could create an optimization. But a few things I see that would have to be guaranteed are

  • Published events are consumed in correct order
  • Published events that were "missed" by a service need to be recognized and somehow compensated for
  • There is no bug in the delta algorithm

whereas if the entire state is published then

  • If 2 published events are consumed in the wrong order then publishing a 3rd event corrects everything
  • If a published event fails to be consumed then publishing a 3rd corrects everything
  • No delta algorithm, so less complexity and compute resource on the publishing service

A few solutions I can think of to make the delta approach a little more robust add a lot of complexity:

  • Publish both the changes and the entire state and a pointer to the previous event. The services consuming the event can figure out whether to process the delta or the whole data set.
  • "Cycle" between publishing deltas and entire state. Maybe publish changes 9/10 times and then the entire state the other 1/10 of the time just to help correct things.

Both of those feel weird. I am interested in whether there exists a robust solution for notifying of "additions, removals and modifications" while ensuring eventual consistency in the system.

Aucun commentaire:

Enregistrer un commentaire