mercredi 24 août 2016

What is the advantage of a parallel pipeline compared with Task Parallelism?

I often read about the pipeline pattern as a common and helpful pattern in terms of exploiting concurrency. But I wonder if there is any advantage of the parallel pipeline pattern compared with the Task Parallel Pattern.

Suppose we have three stages in a pipeline: A, B, C. When data needs to be processed A takes it, processes it and hands it over to B. When the next data chunk is coming in, the same happens and A and B are working concurrently.

So different stages in the pipeline can be executed in parallel, but when we use three pipelines working in parallel (as in Task Parallelism Pattern), we get exactly the same picture. When two data chunks are coming in one after another, the first chunk is taken by Pipeline 1, the next chunk is taken by Pipeline 2 and both chunks are processed concurrently.

Furthermore I can easily imagine a lot of problems in parallel Pipeline: The Buffer between the stages could block (or overflow), one stage is dominating in terms of processing speed so all stages before the slowest stage have to wait etc...

These problems do not exist in the Task Parallelism Pattern. Additionally, this pattern is more flexible when the chunks are coming in faster than the first stage of the pipeline can process them (or they can be fetched concurrently).

So why should I ever use the parallel pipeline pattern?

Thanks in advance for any ideas!

Aucun commentaire:

Enregistrer un commentaire