So I am building a service that has a bunch of integrations like AWS S3, Databases. We use a config file to set the config for the above integrations. I have individual classes for each of integrations and my main entry point parses all the configs from the file stores it in the main pipeline class and then sends it over to the individual classes when they are used. So the code kinda looks like this
db.py
class DB:
def __init__(self, host):
self.host = host
s3.py
class S3:
def __init__(self, bucket):
self.bucket = bucket
Then there is the pipeline class which does a bunch of tasks sequentially and uses the integration
pipeline.py
class Pipeline:
def __init__(self, db_host, s3_bucket):
self.db_host = db_host
self.s3_bucket = s3_bucket
def run(self):
s3 = S3(self.s3_bucket)
s3.upload()
db = DB(self.db_host)
db.upload()
The main.py
file parses the config file and then initializes the Pipeline
class with all the configs which then makes individual integrations objects with their configs. Something like this. main.py
config = parse_config()
Pipeline = Pipeline(config.db_host, config.s3_bucket)
pipeline.run()
This way of doing things this way seems very weird because I am parsing all config at first then passing it to Pipeline which is responsible for doing tasks and then passing individual config to class when they are needed. Is there a better way of passing config to individual config to classes so that I do not have to pass a bunch of config to Pipeline? What are some good design choices here ?
An important thing to consider here is that there could be multiple configs for a particular integration. So I might need to use two DB configs for two DBs to do separate tasks
Aucun commentaire:
Enregistrer un commentaire