lundi 21 août 2023

Database for event driven system with multiple publishers

I am working on an event-driven system, we have a service that subscribes to 2 different event types, each coming from a separate service. In my service, I must combine them and emit the data again as an event. I am using typescript along with Postgres as a database. The rate of incoming events is relatively high and the records keep on updating. I have tried optimist(using version number) and pessimist locking(row-level lock).

With optimist locking there was a delay introduced and with pessimist locking there were many deadlocks. I was thinking to go with some lock-free design, but with that, I need to have many joins and the solution becomes more complex.

enter image description here

Event A

{
order_num:'123'
order_value:1000
route_code : 'r1'
}
{
order_num:'456'
order_value:450
route_code : 'r1'
}

Event B

{
route_code:'r1'
route_paths:[{
path_id:'p1'
path_start:'abc'
path_end:'xyz'
},
{
path_id:'p2'
path_start:'def'
path_end:'pqr'
}]
}

Event S Output

{
path_id:'p1'
path_start:'abc'
path_end:'xyz'
orders:[{
order_num:'123'
order_value:1000
},
{
order_num:'456'
order_value:450
}]
}


{
path_id:'p2'
path_start:'def'
path_end:'pqr'
orders:[{
order_num:'123'
order_value:1000
},
{
order_num:'456'
order_value:450
}]
}

There may be more services in future that Service S will subscribe to, so just wanted to design in a way that complexity is minimum.

I am not sure what is a better approach here. If I am doing something wrong please let me know.

Aucun commentaire:

Enregistrer un commentaire