I have a small(ish) MongoDB collection of 100 documents. Each of these has some fields, which are used in the API call (location, timezone, etc.). Each of these belongs to a profile based on their expected volume, and would dictate how often the API should be called, for that specific item, for example:
- Profile A: API call every 5 minutes if current local time is between 0700 and 1900, otherwise every 30 minutes
- Profile B: API call every 15 minutes, if 0700-1900 as above, otherwise every 30 minutes
- Profile C: Every 30 minutes
During testing, I would call the API for each of these, but now I want to be more specific. What would be an efficient way of doing this(if this were to be hosted on the cloud, I don't want it to loop constantly multiple times per second)?
- Easiest would be a loop that constantly checks
- My gut is telling me to use the python scheduler? Example below:
def update_one(id):
make_api_call(id) #gets the data based on the id specified
process_response(id) #do whatever needs to be done with the information
current_local = get_local_time(id) #get the local time for this place
#find the next time this location is to be polled in the API
next_task = get_next_call(id,current_local)
add_to_queue(id,next_task)
Or, should I rely on existing tools from cloud providers to schedule these things?
Aucun commentaire:
Enregistrer un commentaire