I have run into a problem when designing my software.
My software consists of a few classes, Bot, Website, and Scraper.
Bot is the most abstract, executive class responsible for managing the program at a high-level.
Website is a class which contains scraped data from that particular website.
Scraper is a class which may have multiple instances per Website. Each instance is responsible for a different part of a single website.
Scraper has a function scrape_data() which returns the JSON data associated with the Website. I want to pass this data into the Website somehow, but can't find a way since Scraper sits on a lower level of abstraction. Here's the ideas I've tried:
# In this idea, Website would have to poll scraper. Scraper is already polling Server, so this seems messy and inefficient
class Website:
def __init__(self):
self.scrapers = list()
self.data = dict()
def add_scraper(self, scraper):
self.scrapers.append(scraper)
def add_data(type, json):
self.data[type] = json
...
# The problem here is scraper has no awareness of the dict of websites. It cannot pass the data returned by Scraper into the respective Website
class Bot:
def __init__(self):
self.scrapers = list()
self.websites = dict()
How can I solve my problem? What sort of more fundamental rules or design patterns apply to this problem, so I can use them in the future?
Aucun commentaire:
Enregistrer un commentaire