mercredi 17 avril 2019

How to dynamically load python modules and call method on each of them?

Python beginner here. Let's say I have three methods for scraping websites. Let's call them scrape_site_a, scrape_site_b, and scrape_site_c. I want to run each of these but I'd like to define them in such a way that I can call them dynamically without calling each by name. Ideally I'd like to just load all modules in a directory and call the same method on each of them. My attempt so far is the following:

site_a.py

def scrape():
    # scrape the site

site_b.py

def scrape():
    # scrape the site

site_c.py

def scrape():
    # scrape the site

I have the __init__.py setup such that I can do the following:

scrape.py

from sites import *

site_a.scrape()
site_b.scrape()
site_c.scrape()

I would like to do something like:

for site in sites:
    site.scrape()

I realize that there is a fundamental programming concept I'm not understanding here and I have two questions:

  1. Is there a way to do this using the approach I'm taking?
  2. Is there a better approach? Why?

Aucun commentaire:

Enregistrer un commentaire