dimanche 1 octobre 2017

Is moving large data between Service Fabric services an antipattern?

I'm developing a web data crawler using Service Fabric. I've devided a crawling proccess into Stateless services pipeline. First service loads HTML. (S1) Second one parses data from it. (S2) Third one saves result into DB and performs additional operations. (S3) HTML code moves from S1 to S2, than to S3 (by design, I need HTML in third service). The problem that HTML itself is a big amount of data and I am forced to set

[assembly: FabricTransportServiceRemotingProvider(MaxMessageSize = 1073741824)]

Is this design an antipattern? Should I perform all work with HTML inside singe service?

Aucun commentaire:

Enregistrer un commentaire