API sync requests for immediate response, but we have long running (5 to 100 minutes) jobs where we cannot depend on sync request, we should implement worker nodes handles long running jobs and api just submit jobs and get status.
while I am browsing, recommended way to use cache system , queue system, worker nodes similarly like below diagram
Since Redis is cache system and writing status states relying on redis cache is good approach ?
Another approach is using persistent no sql db, queue system and worker nodes like below diagram
Above approach slows down the submissions and getting status because no cache system is using.
approach 3 I am consider to go with cache no sql and persistent no sql DB similarly like below
I am designing first time production ready application and implementing my. I want to make sure correct design principals are used.
Load estimated per second 1000 requests average
Also how difficult implementing approach 3 ?
Below is what I am expecting.
- Performance
- tasks status api states for at least last 30 days jobs.
Programming: Python Framework: Flask
Aucun commentaire:
Enregistrer un commentaire