lundi 12 juillet 2021

Golang, network application design, sockets, I/O, timers

I am in the process of learning Golang, coming from a decade of developing with C++ and Python.

At work, we mainly build applications that handle protocol transformations. Receive some data on one or more sockets, transform it, hit the filesystem, do stuff, and finally make the data avaialable on another network socket. Also in reverse, making the data available on a REST API for example. A lot of the protocols use timers for timeouts, repeated transmissions, e.t.c.

Using C++ and Python, I usually solve the complexity of all these input/output systems by using a reactor framework (ACE, then moved to asyncio. Twisted in python). The reactor framework takes away a lot of the synchronization issues, as I simply have to think about the program as a single thread of execution, with "tasklets" being scheduled in by a single poll/select loop, buried in the reactor framework. No need for mutexes or atomic queues, whatever, as there only is the one thread. The reactor has methods for registering callbacks on socket i/o, file i/o, timers, e.t.c. so the main program just consists of callback functions, guaranteed to be called on the same thread.

On the rare occasion that I need to do some heavy background work, I can spawn up a thread (std::thread in c++), and schedule the completed work back to the reactor thread (in asyncio, with the post() function for example).

How would I go about solving these kind of problems in Go? or is it the wrong tool for the job?

I have started wrapping my head around channels and goroutines, and it seems like there would be a lot of coordination needed. For example, open a socket. If that does not work, timeout after x seconds, attempt to reconnect. If it opens, receive messages. Decode and demultiplex those messages to other parts of the code, while sending repeating heartbeats out the same socket. This would require a lot of golang select'ing to keep track of, while those selects would have to be torn down and built up again, as the need for heartbeat/timeout timers change, depending on the state of the connection. This is just an example to illustrate a point.

It may be survivor-bias in full effect. when all you know are the tools you already have, every problem will be bent to fit those tools.

Can anyone point me in the direction on some good sources on handling these kind of issues in a go program?

Aucun commentaire:

Enregistrer un commentaire