I have a mongo database and two collections A
and B
, I want to check if record with A.id
exists in collection B
, and if it exist I Want to do some stuff. I need to do this over and over again. So I want to use simple SpringBoot app with @EnableScheduling
and @Schedule
.
It is quite easy if we will have few records in both collections, but I need to process large amount of records in one run it should be about 5000 and task shoul be repeated in 30 seconds cycles.
I was wondering is there any smart solution for this case? How should this be designed to support scaling? - I cant get db.A.find()
because this would give more on less the same results on every instance.
What with multithreading? My only idea is to limit the mongo query db.A.find().limit()
to let say 100
and create ThreadPoolExecutor
with blocking queuq
also set to 100
, but I think this is not a good solution.
Aucun commentaire:
Enregistrer un commentaire