dimanche 27 septembre 2020

Versioning of pipeline elements

I have developed a custom-made Python package, which provides 2 classes to play with: Stage and Step. They operate based on a Chain of Responsibility design pattern, and behave in a way where:

  • Both Stage and Step possess handle(request) method
  • Stage has a list of Stages or Steps underneath it
  • Step is the endpoint, which contains logic to be executed on the incoming request

In case of Stage, executing its handle(request) method will make it iterate over children elements in its list, and execute their handle(request) method while passing request argument in-between them (updated with each element it passes through).

In case of a Step, I have an abstract method logic(request), which needs to be overwritten by the user when inheriting from this class, in order to introduce some logic it needs to apply to incoming request argument.

Pipeline logic

Now, I would like to introduce some sort of version control mechanism inside of this pipeline, so that pipeline can identify itself with some unique marker, unique to the set of version and list of steps included in it. I need it, so that if let's say I introduce some new steps into the pipeline in the future, I would like to be able to e.g. redo so older requests, which have been done with an older version of the pipeline.

I have been wondering about what sort of mechanisms I could leverage to get this done, but so far my only idea was to tag each step & stage with version attribute, like self._version = "1.0", plus have the running script actually count number of steps+stages inside of the pipeline from Stage 0's perspective, and somehow combine it with versions sum. While simple, this would require me to remember to up-rev version attribute on each step I would rework etc., and would be prone for an error.

I was wondering, if there are any alternative methods of doing something like this?

Aucun commentaire:

Enregistrer un commentaire