I'm actually building a streaming module in my system that read and write from/to Kafka. It's done using Spark Streaming. My need is to structure the code in a clean and modular way: I have several chained steps and I would like to compose them. Is there a library that already gives the right abstractions for Spark Streaming? Otherwise, I would like to read something on a design pattern that solve this problem and I could write the abstractions myself.
(Then, is it just me or good resources about idiomatic design patterns in Scala are hard to find on the internet?
Aucun commentaire:
Enregistrer un commentaire