samedi 25 juillet 2015

Structural pipeline in Spark Streaming

I'm actually building a streaming module in my system that read and write from/to Kafka. It's done using Spark Streaming. My need is to structure the code in a clean and modular way: I have several chained steps and I would like to compose them. Is there a library that already gives the right abstractions for Spark Streaming? Otherwise, I would like to read something on a design pattern that solve this problem and I could write the abstractions myself.

(Then, is it just me or good resources about idiomatic design patterns in Scala are hard to find on the internet?

Aucun commentaire:

Enregistrer un commentaire