samedi 28 octobre 2023

Batch workloads vs calls to microservices

This is quite a general question. I am quite new to all of this and I am having trouble deciding if something should be considered a batch job or a simple request to a microservice.

Let's say users can upload 30-second videos to S3 and we want to, for some reason, process every video such that the individual images of the video are extracted and stored in S3 again. Also, we do not really care about the latency, and minimizing compute costs, i.e., the cloud bill, is more important.

Would you use something like AWS Batch? Or would you deploy a set of microservices that do the processing by responding to API service calls, possibly with a messaging queue between S3 and the microservices?

Both options seem to do the job, but what is the right or correct way to approach such a problem?

I have read over https://docs.aws.amazon.com/batch/latest/userguide/best-practices.html. For short jobs, it says that you would need to merge jobs such that they run, ideally, 3 to 5 minutes each. However, it does not really talk about what should not be considered a batch job.

I feel like this question should be googleable, but I might just be missing the right terminology here.

Aucun commentaire:

Enregistrer un commentaire