mercredi 14 septembre 2022

NodeJs - How to handle broken stream reader piped from a stream writer?

I have an application that needs to read from a TCP socket and write to a database. It is known that, the rate of reading from TCP socket and writing to database will be different. To decouple the read and write, I am introducing a Duplex stream. Therefore, the flow is expected to be as follows.

  1. Read from TCP socket.
  2. Write to Duplex stream.
  3. Read from Duplex stream.
  4. Write to database.

The question: What is the "right" thing to do when the database connection drops? The following options exist.

  1. Handle database connection drop and terminate application gracefully.
  2. Automagically redirect reads from Duplex stream (Step 3) above to a file so that, when database reconnection happens, the file and stream are processed. The database can tolerate "past" data inserts. The application is not terminated.

Option 1 is easy but losing the TCP stream data is "expensive" in my use case. Option 2 does not lose TCP stream data but sounds complicated to implement.

Can you please advise what is a "best practice"? Or, how do we implement Option 2 (at least, the redirection part)?

Aucun commentaire:

Enregistrer un commentaire