vendredi 27 mai 2022

Python and Unix pipes

I have an interesting design problem and I wanted to ask the community for advice. I am designing a bunch of tools that deal with molecules, which are handled as a pickable class. The tools adhere to the KISS principle, so each is going to be specialized in one task only, and the idea is to allow to optionally combine them using Unix pipes to do something like this:

$> generator.py | filter.py [OPTS] | calc_properties.py

While each tool can be used independently, by allowing chaining multiple tools reduces redundancy and the amount of data to be saved and read at every step. Also, each tool can potentially generate more than enough data to saturate all the available RAM. I've looked around and while there is quite a lot of documentation about Python support for pipes, I can't find anything that tells me first if this is a good design pattern, and second what would be a good way to pass around pickable objects through the pipes.

Aucun commentaire:

Enregistrer un commentaire