mardi 11 février 2020

How to perform retriable non transactional service orchestration via KAFKA

Hi I have a problem figuring out how a classic SOA scenario will project over KAFKA processing.

I have the following case. I have an Order this order letys say contains several sections. User Data, Technical Data and Specification. When a change or create event is recieved a copy of the complete order is send to the integration layer. What I need is to perform an orchestration upon 3 services SpecificationService , TecnicalDataService and Customer Service. Customer service and TechynicalDataService can be executed independently of each other but only after SpecificationService has completed successfuly.

If we convert the problem into classical worktable solution. This would mean a worktable with following columns OrderID | Time of event | IsProcessed| isSpecificationSent| isCustomerdataSent| is technicalData sent

What I was thinking to do with KAFKA is to have 3 topics:

  1. OrderEvent topic that
  2. retry topic
  3. Successfully processed topic

I was thinking of two directions 1)Use Consumer/Producer api 2) Try to use Streams and Processing API.

The problem is that If event for purchase order lies on the event topic, than it is possible that there is a second event on the same order with later chronological order that is also available. So if I fail one order event I need to force fail all subsequent events of this order.

I was thinking to use StateStore that is shared accross two streams the Event procesing stream and the retry topic processing stream. But from what I see this is not possible.

How can I organize my processing / service orchestration around KAFKA and not around a worktable in the DB ?

Aucun commentaire:

Enregistrer un commentaire