lundi 31 octobre 2022

Design pattern for file reader?

I am creating a module that reads CSV and Excel files using Apache POI and Opencsv.

For reading CSV files, I am creating 1 class and 2 methods:

class CsvReader {
  void open() {//implementation}
  List<CsvDto1> get1() {//implementation}
  List<CsvDto2> get2() {//implementation}
  void close() {//implementation}
}

For reading Excel files, I am also creating 1 class and 2 methods:

class ExcelReader {
  void open() {//implementation}
  List<ExlDto1> get3() {//implementation}
  List<ExlDto2> get4() {//implementation}
  void close() {//implementation}
}

All I want is to implement a pattern that will be helped maintainable in the future. So I created an interface called FileReadable:

interface FileReadable {
  void open();
  List<CsvDto1> get1()
  List<CsvDto2> get2()
  List<ExlDto1> get3()
  List<ExlDto2> get4()
  void close();
}

then CsvReader and ExcelReader will be implemented from FileRedable. The issue is get1() and get2() exist in CsvReader but they do not exist in ExcelReader, get3() and get4() exist in ExcelReader but they do not exist in CsvReader. How do I create a common read method for both classes or do we have any design pattern for this case?

   interface FileReadable {
      void open();
      Reader read();
      void close();
   }

Two singletons or Two members of a singleton?

In a C# app, a thread-safe singleton can be implemented by Lazy<T>.

In my app (regarded as a system), if I would like to have two or more global settings (a & b) which also need thread-safe, which solution is better?

  1. One singleton has two settings:
class A
{
    public int a { get; set; }
}

class B
{
    public int b { get; set; }
}

class Singleton
{
    static readonly Lazy<Singleton> instance = new Lazy<Singleton>(() => new Singleton());
    public static Singleton Instance { get { return instance.Value; } }

    Singleton()
    { }

    public A A { get; } = new A();
    public B B { get; } = new B();
}

static class Test
{
    public static void Main()
    {
        TaskFactory.StartNew(() =>
        {
            while (true) Singleton.Instance.A.a = 1;
        });
        TaskFactory.StartNew(() =>
        {
            while (true) Singleton.Instance.B.b = 2;
        });
    }
}
  1. Two different singletons:
class SingletonA
{
    static readonly Lazy<SingletonA> instance = new Lazy<SingletonA>(() => new SingletonA());
    public static SingletonA Instance { get { return instance.Value; } }

    public int a { get; set; }
}

class SingletonB
{
    static readonly Lazy<SingletonB> instance = new Lazy<SingletonB>(() => new SingletonB());
    public static SingletonB Instance { get { return instance.Value; } }

    public int b { get; set; }
}

static class Test
{
    public static void Main()
    {
        TaskFactory.StartNew(() =>
        {
            while (true) SingletonA.Instance.a = 1;
        });
        TaskFactory.StartNew(() =>
        {
            while (true) SingletonB.Instance.b = 2;
        });
    }
}

My current implementation is Solution 1.

Importing data from different sources and merge into one

We are fetching data from different ERP sources and creating the same entities in our system. Every ERP system has different properties, I convert all these entities into a single general entity. I want to write a service that will receive data from all these different sources and convert their data into a single entity based on their type, either from oracle or syspro or sage. I think the decorator design pattern will be perfect to overcome this issue. what you say

dimanche 30 octobre 2022

How do Laravel route groups work under the hood?

I was wondering how is this working under the hood:

Route::group(['prefix' => 'parent'], function() {
    Route::get('/child', 'MyController@myMethod');
});

I don't understand how is Laravel associating the routes definitions in the anonymous function to the parent group.

Something like this would have made much more sense:

Route::group(['prefix' => 'parent'], function($group) {
    $group->get('/child', 'MyController@myMethod');
});

I tried diving into what happens under the hood through a debugger, and I noticed that in Illuminate\Routing\Router.php this happens:

protected function loadRoutes($routes)
{
    if ($routes instanceof Closure) {
        $routes($this);
    } else {
        (new RouteFileRegistrar($this))->register($routes);
    }
}

The anonymous function that contains our routes definitions is called and $this gets passed to it, altho our anonymous function doesn't accept any argument, so why passing $this to it?

Python Watcher/Observer over opened/closed applications

I am trying to build an observer for the system to watch for applications' close/open/minimized/activated events

I was able to build the below observer. However, I was stuck on how to connect it

class AbstractObserver(ABC):
    """ The Observer interface declares the update method, used by subjects. """

    @abstractmethod
    def update(self, ) -> None:
        """ Receive update from subject. """
        pass


class AbstractObserved(ABC):
    """ The Subject interface declares a set of methods for managing subscribers. """
    _state = None
    _observers = {}

    @abstractmethod
    def attach(self, id) -> None:
        pass

    def detach(self, id) -> None:
        """ Detach an observer from the subject."""
        print(f'Task{id} Detach From Observer.')
        try:
            self._observers.pop(id, None)
        except ValueError:
            print(f'Task{id} Not found in observers.')
        pass

    @abstractmethod
    def notify(self, ids, state='create') -> None:
        """ Notify all observers about an event."""
        pass

class ApplicationObserver(AbstractObserver):
    _tracker = None
    _start = None

    def update(self, ) -> None:
        pass


class ObservedApplication(AbstractObserved):
    """ The Subject interface declares a set of methods for managing subscribers. """
    _state = None
    _observers = {}

    def attach(self, id) -> None:
        """ Attach an observer to the subject. """
        pass

    def notify(self, ids: dict, state='create') -> None:
        """ Notify all observers about an event."""
        pass

I am trying to connect it to the generator results from psutil.process_iter() function But, as a generator, there is always an extra step of converting it to a list causing me troubles in the connection

Select-string patern match and notmatch

I am fairly new to Powershell and have run into a bit of a pickle.

I'm trying to find string "o_dwh" in scripts, but if there is exec before this statement like this - "exec o_dwh" - I don't want to select that. How do I do that?

So far I have this:

Get-ChildItem -Path "$packagepath\Scripts\" -Include *.txt -Recurse | Select-String -Pattern "o_dwh"

I tried this, but I know it's wrong:

Get-ChildItem -Path "$packagepath\Scripts\" -Include *.txt -Recurse | Select-String -Pattern "o_dwh" and "exec o_dwh" -notmatch

samedi 29 octobre 2022

Main Differences between static class and singleton pattern?

What real (i.e. practical) difference exists between a static class and a singleton pattern?

Both can be invoked without instantiation, both provide only one "Instance" and neither of them is thread-safe. Is there any other difference?

Both can be invoked without instantiation, both provide only one "Instance" and neither of them is thread-safe. Is there any other difference?

python singleton pattern request with different session for different services

I am using the python singleton pattern to create an API client and doing API authentication on a request session to reuse existing authentication. The current approach working fine but I need slightly change my current approach to support multiple services from the same API client. I am attaching my existing working code below-

class Singleton(type):
    _instances: Dict[str, str] = {}

    def __call__(cls, *args, **kwargs):
        if cls not in cls._instances:
            cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
        return cls._instances[cls]

class ApiClient(metaclass=Singleton):

    def __init__(self):
        self._bearer_token: Optional[str] = None
        self._token_expires_at: Optional[datetime] = None
        self._session = requests.Session()
        self.access_token_url = 'token_url'

    @property
    def is_client_authenticated(self) -> bool:
        if not self._bearer_token or not self._token_expires_at:
            return False
        return datetime.now() < self._token_expires_at

    @property
    def session(self) -> requests.Session:
        if not self.is_client_authenticated:
            self.authenticate_client()
        return self._session

    def authenticate_client(self) -> None:
        url = self.access_token_url

        payload = {
            "client_id": CLIENT_ID,
            "client_secret": CLIENT_SECRET
        }

        response = self._session.post(url, data=payload)
        self._bearer_token = response["access_token"]
        auth_header = {"Authorization": f"Bearer {self._bearer_token}"}
        self._session.headers.update(auth_header)

    def get_data(self,id):
        url = f"http://example.com/{id}"
        params = {
          "year": year,
          "month": month,
        }

       response = self.session.get(url, params=params)

After that, I'm creating an instance of singleton ApiClient to call the existing method of the class.

service = ApiClient()
service.get_data()

But now I want to inject some variables while creating the ApiClient instance so that I can create multiple services with singleton patterns with different initialization values(Need to be sure I am using the same session for the same service because I want to reuse the session for that service.)

service_a = ApiClient(client_id_a, client_secret_a)
service_a.get_data() 

and

service_b = ApiClient(client_id_b, client_secret_b)
service_b.get_data() 

Note: I have the plan to modify the Singleton class to take those client_id and client_secret and create instances based on service instead of class, So I may need to modify this like below, Now I want to get any other advice or suggestion on how can I do this with fewer changes on my existing codebase.

 cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)

vendredi 28 octobre 2022

Different interfaces in Typescript Factory

I have a factory like that:

@Injectable()
export class JobApiFactoryService
  implements ApiServiceFactoryInterface<JobApiPayloads>
{
  public constructor(
    private readonly createApplicationActivityService: CreateApplicationActivityService,
    private readonly createActivityService: CreateActivityService
  ) {}

  public async execute(
    payload: JobApiPayloads,
    action: ApiNamesEnum
  ): Promise<void> {
    switch (action) {
      case ApiNamesEnum.CREATE_JOB_APPLICATION_ACTIVITY_EXTENSION:
        await this.createApplicationActivityService.execute(payload);
        break;
      case ApiNamesEnum.CREATE_JOB_ACTIVITY:
        await this.createActivityService.execute(payload);
        break;
      default:
        throw new Error("Invalid action");
    }
  }
}

JobApiPayloads:

   export type JobApiPayloads = JobApplicationActivityExtensionPayloadInterface 
  JobActivityExtensionInterface;

createApplicationActivityService it's using JobApplicationActivityExtensionPayloadInterface as type for payload

createActivityService it's using JobActivityExtensionInterface as type for payload

I get this error:

Argument of type 'JobApiPayloads' is not assignable to parameter of type 'JobApplicationActivityExtensionPayloadInterface'.   Type 'JobActivityExtensionInterface' is missing the following properties from type 'JobApplicationActivityExtensionPayloadInterface': jobId, candidate_ids

Regular Expression to match on common starting characters including wildcard characters in either/both test and lookup strings

I'm trying to construct a regular expression to match an input/test String variable against a list of strings. The logic needs to be as follows :

  1. If there is an item in the list that is simply a single asterisk "*", consider that a match

  2. If there is an item in the list where the entirety of the string matches the beginning of the test string, consider that a match

Test String List Item Expected Result
ABCD A Match
ABCD AB Match
ABCD ABC Match
ABCD ABCD Match
ABCD ABCDE Not a match
ABCD AC Not a match
ABCD ABD Not a match
ABCD XA Not a match
ABCD XABC Not a match
  1. If there is an item in the list which contains one or more period characters ("."), they should each be treated as a single-character wildcard (equivalent of "?" or could also be [A-Z]{1})
Test String List Item Expected Result
ABCD A.C Match
ABCD AB.D Match
ABCD A.CD Match
ABCD A..D Match
ABCD A.B Not a match
ABCD A.BC Not a match
  1. Similarly, any period characters in the test string should also be treated as single-character wildcards
Test String List Item Expected Result
A..D ABCD Match
A..D AB.D Match
A..D A.CD Match
A..D A..D Match
A..D ABC Not a match
A..D ACBD Not a match
A..D A.BC Not a match

(NB : Period characters cannot appear at the very start or very end of either the test string nor any of the list items - only surrounded by alpha characters)

So, taking the "ABCD" example, a (poor) regular expression that (I think?) would work would be something like :

^((\*)|(A)|(AB)|(ABC)|(ABCD)|(A\.CD)|(AB\.D)|(A\.\.D))$

Regular expression visualization

Debuggex Demo

The "A..D" example is slightly more straightforward (again, I think?) :

^((\*)|(A)|(A.)|(A..)|(A..D))$

Regular expression visualization

Debuggex Demo

However - the test string is dynamic (string variable) so that would mean I would have to construct this pattern with some kind of loop or nested loops based on the characters in the test string every time I need to run a pattern match. Which is fine if the test string is short like "ABCD" but the pattern complexity grows exponentially as the length of the test string increases.

For example, if "ABCD" is changed to "ABCDE", the equivalent pattern becomes :

^((\*)|(A)|(AB)|(ABC)|(ABCD)|(ABCDE)|(A\.CDE)|(AB\.DE)|(ABC\.E)|(A\.\.DE)|(AB\.\.E)|(A\.\.\.E))$

Regular expression visualization

Debuggex Demo

So... I'm just wondering if there's a smarter way of constructing a regex pattern that meets these rules, based on an input/test String variable of arbitrary length?

I print star pattern but output is showing vertically why this happened

As shown in picture, what's the mistake?

I tried maximum time

jeudi 27 octobre 2022

Sending Condition Varible Signal From Abstarct Base Class to Derived Classes

There is an Abstract Base Class with a static function and and a static variable and with a pure virtual function.

For Each Request update the static variable using static function. And for each update It should be notify the base class condition variable each of them are different from each other.

class Base{
   protected:
      static int ms_Varible;
      static std::mutex m; 
   public:

       static void update(int pVar){
          m.lock();
          ms_Varible = pVar;
          m.unlock();
          if(ms_Varible == 1){notifyDerived1()}
          if(ms_Varible == 2){notifyDerived2()}
          if(ms_Varible == 3){notifyDerived3()}
       }

       std::string toString() = 0;

};


class Derived1 : public Base{

   public:

      void foo(){

        while(true){
        
             wait_cond(cond_varible);
        }

      }

      std::string toString(){return "Derived1";}
};

....

In this scenario where should I keep the condition variables. How Can I notify them in base class.

Is the design could be better? I think the condition variable can be static in the derived classes but it does not seems to right.

Thank you for your time.

Is it better to encapsulate variables in an object before passing it by reference to a function ? which design to pick ? why? [closed]

I'm often facing the same designing issue.

Should I make a single object holding multiple members then pass it to a function or should I just pass each parameter individualy to the function ?

Usualy, I like to pass a single compound object to a function because.

  1. I feel it makes the code more readable.
  2. If I need a new parameter (I often need), I can add a new member to the object, and the function declaration stays unchanged. So my code doesn't break. Object constructor makes the job to avoid undefined behavior inside the function.
  3. I can export the function inside a dll, the client will have nothing to change if my object declaration has changed. unless I want to, I can change the constuctor, so we know a new param is expected.
  4. The execution is faster, Instead of passing every primitives to the function, I just pass a single adress.
  5. There must be many many more reason linked to the OOP I could not describe. And that's where I probably need the help.

One downside, is sometimes I tend to mix socks with apples. I make a function with 5 parameters, then later I need a 6th parameter. so I make a compound object with 6 members. but another 7th parameter comes...and has nothing to do with the class I did before. instead of having issue with my function declaration now it's my class which comes heavy.

What are the best practices ? Where do I start to learn ? Is there any good reasons to keep passing a lot of multiple parameters to a function ? Does tending to reduce the number of parameter as much as possible a good practice ?

There's 3 approaches that quickly comes to my mind.

What should I asked to myself to decide which approach is the best ?

// approach I
float computeWeight(const float &a, const float &b, const float &c, const float &d) {
    return a+b+c+d;
}

int main(){
   
    float a = 0.0;
    float b = 1.0;
    float c = 2.0;
    float d = 3.0;

    float result = computeWeight(a,b,c,d);
    return 1;
}

// approach II

struct bunchOfFlt{
    float a;
    float b;
    float c;
    float d;
} 

float computeWeight(const bunchOfFlt & myBunch) {
    return myBunch.a + myBunch.b + myBunch.c + myBunch.d;
}

int main(){
    bunchOfFlt myBunch = {0.,1.,2.,3.};
    float result = computeWeight(myBunch);
    return 1;
}


// approach III 

struct bunchOfFlt{
    float a;
    float b;
    float c;
    float d;

    float computeWeight() const {
         return a + b + c + d;
    }

} 


int main(){

    bunchOfFlt myBunch = {0.,1.,2.,3.};
    float result = myBunch.computeWeight();
    return 1;
}

What is the best approach here, is there any one faster than the other ? Which one to pick should be obvious. Sometimes depending of how the program may evolve or not, I feel very lost when I'm designing new programs and It's time for me to learn a bit of theory.

Is it good practice in OOP to make Children capable of self-deleting from it's Parent?

Let's say we have 3 classes: ChildFromApi, Child (for sake of simplicity, hydrated ChildFromApi) and Parent. There is relation between them:

Child {
  parent: Parent;
  childFromApi: ChildFromApi;
  
  //should remove this particular child from parent's childrenFromApi and children lists
  delete(): void;

  constructor(parent: Parent, childFromApi: ChildFromApi)
}

Parent {
  childrenFromApi: ChildFromApi[];
  //mapped version of childrenFromApi
  children: Child[];

  constructor(childrenFromApi: ChildFromApi[])
}

Is it good practice in OOP to make in Children methods like delete() that are capable of mutating Parent? Or should I better create method deleteChild(child: Child): void in Parent?

I am a frontend developer and from perspective of my tasks, allowing children to self-delete seems very convenient

Best way to handle infrastructure code inside objects

I am working on a solution that interacts with several external API's. The basic structure of the code looks something similar to:

  • Some basic types that encapsulates the parameters needed to interact with the APIs (for example, resource names, status, ids...). Those types are infrastructure agnostic, so they are only defined in terms of primitive types and without any methods.
  • API connectors, which are responsible of transforming the previous types into valid arguments.
  • And last, a class that acts as a manager, who reads the serialized types, performs some logic over them and delegates the corresponding actions to the connectors (create resource if not exists or update it).

The key point that I am trying to deal with appears when in some specific cases, the API connectors needs to upload an image. To perform the upload we need to send it as bytes, but in our types we are modeling that images with urls (which could be an S3 route, local one, etc).

  • If we change the core types to include the bytes of the image, we will be carring the entire image for every step of the process, no matter if we need it or not.

  • On the other hand, if we delegate the operation of reading the image to bytes to the connectors, we will we introducing pieces of infrastructure in the connector and we dont see it as a good practice due to the coupling (we need to keep the connectors as much decoupled as possible).

  • And last, if we include the path to bytes operation in the manager, it gets much complicated as it seems because is the connector who answers the question 'do we need to upload the image?'.

I am wondering if there is some software pattern I am missing to solve that or maybe is not possible to keep the manager simple when considering that situation.

mercredi 26 octobre 2022

Builder Design Pattern - How to make field only accessible to one of its subtype

I have created builder for client type A and B. Client TypeA has two fields(see below), while client TypeB has two additional fields. How can I make sure field C and D are only accessible to client TypeB, i.e. client TypeA should not take field C and D when creating.

  client.builder()
    .withClientType(ClientType.TypeA)
    .withFieldA(fieldA)   
    .withFieldB(fieldB)
    .build();
  client.builder()
    .withClientType(ClientType.TypeB)
    .withFieldA(fieldA)   
    .withFieldB(fieldB)
    .withFieldC(fieldC)
    .withFieldD(fieldD)
    .build();

What's the correct usage builder pattern in this case?

When/where is the best moment to check for Firestore document ownership (service or middleware)

When implementing security rules to ensure a user can only fetch/update their own documents in Firestore, where should I put the logic?

(Before you say 'in Firestore Security Rules file', please note that I am using the Firebase Admin SDK which bypasses these security rules rather than the frontend Firebase SDK.)

For more context:

  • I have built a CRUD REST API using a Koa server implemented in Firebase Cloud Functions which documents stored in Firestore.
  • API calls are authenticated using Firebase Auth, and I decode the authorization token to get the User ID of the person calling it.
  • I created a FirestoreService class which can fetch and read data from Firestore.
  • I use custom Koa middleware to check the user has relevant roles and permissions (uses AccessControl library) but this middleware doesn't check for document ownership, only that someone has the permission to 'getOwn' or 'getAny'.

But where should I put the logic to check, for example, that a document being fetched has a 'userId' property set to the ID of the currently logged in user.

It seems I have 3 options:

  1. In middleware. Before passing responsibly to controller to do the work, middleware should perform a 'get' operation then check the userId attribute of the document returned matches the logged in user. Downsides of this approach is that I am fetching the document twice - once to check permissions and then again in the controller.

  2. In controller. Once the data has been fetched from the database I do the userId check. Downsides of this are code replication as I'll have to write this logic in each controller function.

  3. In service. Perform the check in the FirestoreService class once the data has been fetched. Downside of this approach is that it feels like I'm adding additional logic to a service which should remain generic. The service get method would need to be called with details about what attributes to check and know the userId.

What's the best approach to this puzzle in terms of architecture?

Bear in mind it should work for all CRUD operations, not just the 'read' operation.

Any help much appreciated!

I started with the logic in the service but it feels clunky.

Decompiling the proxy class in proxy design pattern

In Proxy why we want to have 2 classes, when decompiling the proxy class the real class can be visualized. so what is the real use?

Allow pattern destructuring but disallow construction

Suppose I have the following src/lib.rs

pub enum Toppings {
    Almond,
    Caramel,
    Cookie,
    Marshmallow,
    ChocollateSprinkles,
}
pub enum Icecream {
    Chocollate(Toppings),
    Mint(Toppings),
    Vanilla(Toppings),
}

I want to allow pattern destructure s.t. people can use the library like any other enum with associated data:

use Icecream::*;
use Toppings::*;

fn order_how_many(icecream: Icecream) -> usize {
    match icecream {
        Chocollate(Marshmallow) => 42,
        Vanilla(Cookie) => 69,
        _ => 42069,
    }
}

But at the same time I want to forbid creation of certain combinations of enum and its associated data, e.g. maybe the local ice cream store thinks double chocolate is too much and would never sell Icecream::Chocollate(Toppings::ChocollateSprinkles), so we should outright forbid any possibility this combination is constructed. I find this hard to implement in Rust, since I need pub enum to allow pattern destruction, but making the enum public means all of its variants are pub.

I tried private token similar to which sometimes could be found in sealed trait pattern, using private modules and prevents any accidental pub use via #[forbid(missing_docs)], s.t. only crate implementation can decide what Icecream/Toppings combination are possible, but this makes pattern matching ugly (require _ or .. in every pattern destruction).

mod icecream {
    // Intentionally NOT use /// comment for this mod
    // to prevent accidental `pub use`
    #[forbid(missing_docs)]
    mod hide {
        pub struct Hide;
    }
    pub enum Toppings {
        Almond,
        Caramel,
        Cookie,
        Marshmallow,
        ChocollateSprinkles,
    }
    pub enum Icecream {
        Chocollate(Toppings, hide::Hide),
        Mint(Toppings, hide::Hide),
        Vanilla(Toppings, hide::Hide),
    }
    pub use Icecream::*;
    pub use Toppings::*;
}

pub use icecream::*;
fn order_how_many(icecream: Icecream) -> usize { // OK
    match icecream {
        Chocollate(Marshmallow, _) => 42,
        Vanilla(Cookie, ..) => 69,
        _ => 42069,
    }
}
fn str_to_icecream(base: &str, topping: &str) -> Option<Icecream> { // Compile fail as intended
    if base.to_lowercase() == "chocollate" && topping.to_lowercase() == "chocollatesprinkles" {
        Some(Chocollate(ChocollateSprinkles, icecream::hide::Hide))
    } else {
        None
    }
}

Before posting the question SO suggested me this, but it also doesn't seem to fix the problem of enum here, since enum unlike struct cannot have different visibility between the type itself and its associated members, and that this would make having different shape of associated data more cumbersome to implement, e.g. if the length of tuples are different I'd probably have to implement a trait and return trait objects if I were to use this method.

Is there a more elegant/natural/rustacean way to do this? Or one should outright try avoid such code at the beginning, maybe since it's deemed as a bad practice?

mardi 25 octobre 2022

Best way to consolidate state dependent boiler plate in React Functional component

I'm trying to understand if there is a better design pattern to this use case.

I have a react functional component such as this.

const ComponentMajor = (props) => {
   [state, setState]
   classes = useStyles()
   
   //image a whole bunch of other state.
   return (
   <div stateDependentProp1 stateDependentProp2>{stateDependent3}</div>
   <div stateDependentProp1 stateDependentProp2>{stateDependent3}</div>
   <div stateDependentProp1 stateDependentProp2>{stateDependent3}</div>)
   //imagine each of these is maybe 50 lines of code and we need 10 of them.
   //perfect use case for code re use.
}

So naturally I create a div component, but since it's so coupled with ComponentMajor and only needs to be used in ComponentMajor to eliminate code dupl, i want it to live inside the same file.

Option 1

const ComponentMajor = (props) => {
   [state, setState]
   classes = useStyles()
   
   const boilerPlateComp = (props) => <div props>{stateDependent3}</div>
   //image a whole bunch of other state.
   return(
   <boilerPlateComp props/>
   <boilerPlateComp props/>
   <boilerPlateComp props/>)
   
}

This isn't good because every time ComponentMajor re-renders it redeclares boilerPlateComp and has weird behaviors.

Option 2

declare boilerPlateComp OUTSIDE ComponentMajor - but this makes it inefficient to share scope (which if defined inside ComponentMajor is not an issue) between them. It results in some annoying boiler plate ironically to pass the scope dependencies in as props.

The cleanest I was able to come up with was something like this, but there HAS to be a better way to do this in FUNCTIONAL components.

const boilerPlateComp = ({sharedScope, props}) => {
    let {stateDep1, stateDep2, stateDep3, etc} = sharedScope
    return (<div props>{stateDependent3}</div>)
}

const ComponentMajor = (props) => {
   [state, setState]
   classes = useStyles()
   let sharedScope = {state, setState, classes, etc}
   //image a whole bunch of other state.
   return(
   <boilerPlateComp props sharedScope={sharedScope}/>
   <boilerPlateComp props sharedScope={sharedScope}/>
   <boilerPlateComp props sharedScope={sharedScope}/>)

}

My Question Is:

What is the standard design pattern for this common use case? Is this a scenario where a class component would make more sense?

I tried playing around with memoizing the components declared WITHIN ComponentMajor but didn't seem to work.

Is there a clean way to use javascript apply to pass the scope onto the component instances?

Which code is better designed and how to choose

In a game: say there is a treasure hidden behind the stone. When the stone is removed by the player, the treasure will be displayed.

design 1

Stone hold a Treasure

class Treasure
{
    public void SetActive(value: bool) {...}

    public void Init(Stone stone) 
    {
        this.SetActive(false);
    }
}

class Stone
{
    private Treasure treasure; // hold

    public void Remove()
    {
        // ...

        this.treasure.SetActive(true);

        // ...
    }
}

design 2

Stone trigger a event when removed, Treasure ...

class Treasure
{
    public void SetActive(value: bool) {...}

    public void Init(Stone stone) 
    {
        this.SetActive(false);
        stone.OnRemoved += () => { this.SetActive(true); }
    }
}

class Stone
{
    public event Action OnRemoved;

    public void Remove()
    {
        // ...

        this.OnRemoved?.Invoke();

        // ...
    }

}

Which design is better or better?

I'm a little confused and hesitant, are there any good suggestions and rules about these? thanks.

IT architecture template follow ISO standards

Kindly, I am about to develop an IT architecture document for the applications which I am managing, need your help in sharing with me an IT architecture template that follows ISO standards.

And appreciate it if you share an example of one real IT architecture as it will be much helpful.

Note: My expectation from the IT architecture, is that it should show the software and hardware architecture and is less relevant to the overall business and company strategy, but more focused on how the specific solution can be served by our platform.

Thanks ...

Copy internal details from other object with abstract data type

I want a derived class to be able to copy from another class if it has the same type, if not just do nothing.

Example in semi-pseudocode:

class Animal
{
public:
  virtual void CopyFrom(Animal* animal) = 0;

  // TODO: virtual destructor
};

class Dog : Animal
{
public:
  Dog() {}

  void CopyFrom(Animal* animal) override {
    if (animal is Dog) {
      likes_to_bark = ((Dog)animal).likes_to_bark;
    }
  }

private:
  bool likes_to_bark;
}

I know this can be achieved a dynamic cast, but that is considered code smell. I was looking into the visitor pattern and double dispatch, but I could not find a way to do this. Any ideas?

lundi 24 octobre 2022

better way of cancelling subscriptions in angular?

Is there any design pattern in angular 12 to cancel observable subscriptions?

Currently we are developing an angular app and found out that some of our components does not unsubscribe to the observable and hence it is causing memory leakage and some weird behavior in different pages. So instead of closing the subscriptions in each component using ngDestroy like for example below, is there a better way like a design pattern that can be reused across the app without changing existing behavior and minimal code as possible?


import {Component, OnInit, Output, EventEmitter, OnDestroy } from '@angular/core';  
import { HttpClient } from '@angular/common/http';  
import { Subscription } from 'rxjs';  
@Component({ … })  
export class ExampleComponent implements OnInit, OnDestroy {  
 data: any;  
 private subscription: Subscription;  
 API: string = 'https://jsonplaceholder.typicode.com/todos/1';  
  constructor (private http: HttpClient){}  
  ngOnInit() {  
   // Subscribed here  
    this.subscription.add( 
     this.http.get(this.API).subscribe(  
     res => {  
        this.data = res;  
     } ); 
   ) 
 }  
  ngOnDestroy() {  
   // Unsubscribed the subscription  
  this.subscription.unsubscribe();  
  }  
}

how to organize the code when multi varibales get data form multi sources?

I have an API that return a struct which contains variables: a, b and c. To get the value of a,b,c, I have to get original data from multi services,e.g. , S1,S2. S1 returns a1,b1,and S2 returns b2,c2 Now I have to calculate the value of a using a1 and c using c2 and b using b1 and b2. So in this case, how to organize the code? e.g.:

ReqS1();    // here to calculate a and store b1
ReqS2();    // here to calculate b and c

or

ReqS1();  // here to store a1 and b1
ReqS2();  // here to store b2 and c2
CalculateA();  // here to calculate a
CalculateB();  // here to calculate b
CalculateC();  // here to calculate c

or any other better ways?

C++: Composition vs. Multiple Inheritance and Implications for Maintenance [closed]

When considering inheritance for code reuse (i.e. not only for interfaces), multiple inheritance would be necessary for most non-trivial cases. As an alternative, composition is usually preferred. I know that this topic has been discussed extensively here, but I still feel like I did not grasp the aspect of maintenance:

Why is composition considered to be easier to maintain?

Can you give me a concrete example, when and how MI would lead to code that is hard to maintain? I found many answers that describe the problems of MI, but fail to connect how they actually affect code maintenance in a negative way.

Python architecture, correct way to pass configurable initialization object to project modules

I have a big architecture question about how to pass set of configurable / replaceable objects to modules of my project? For example the set may be a bot, logger, database, etc.
Currently I'm just importing they, it make a big problem when you want to replace them during a tests.
Lets' say import app.bot will hard to test and patch

I had tried a multiple solutions but failed with them:

Option 1:
Create some base class with which will accepts set of such objects (db, bot, etc).
Every logic class (who need this set) will inherit this class.
AFAIK the similar approach there in SqlAlchemy ORM.

So the code will looks like:

app.config.py:

Class Config:
    db: DB
    ...

tests.py:

import app.config

Config.db = Mock()

create_app.py:

import app.config

def create_app(db):
    app.config.Config.db = db

logic.py

import app.config

User(app.config.Config):
    def handle_text(text):
        self.db.save_text(text=text)
        ...

The problem with this case is that most likely you can't importing as from app.config import Config
because it will lead to wrong behavior and this is implicit restriction.

Option 2

Pass this set in __init__ arguments to every instance.
(It's ma be a problem if app has many classes, > 20 like in my app).

User:
    def __init__(..., config: ProductionConfig):
        ...

Option 3
In many backend frameworks (flask for example) there are context object.
Well we can inject our config into this context during initialization.

usage.py:

my_handler(update, context):
    context.user.handle_text(text=update.text, db=context.db) 

The problem with this approach is that every time we need to pass context to access a database in our logic.

Option 4
Create config by condition and import it directly.
This solution may be bad because conditions increases code complexity.
I'm following rule "namespace preferable over conditions".

app.config.py:

db = get_test_db() if DEBUG else get_production_db()
bot = Mock() if DEBUG else get_production_bot()

P.S. this question isn't "opinion based" because in some point the wrong solutions will leads to bad design and bugs therefore.

dimanche 23 octobre 2022

draw diagonal pattern using d3.js using typescript

I want to display diagonal patterns for UTR3. To create bars I have created includeSpotn function.

includeSpotn(feature:string, startPosition: number, endPosition: number, yaxis: number,min:number,max:number) {
let x1 = ((startPosition - min)/(max-min))*100
let x2 =  ((endPosition - min)/(max-min))*100
console.log(feature + " : " + this.pickColorn(feature) + " " + startPosition + " : " + x1 + "|| " + endPosition + " : " + x2)
this.svg
  .append('line') // attach a line
  .style('stroke',  this.pickColorn(feature)) // colour the line
  .style('stroke-width', 10)
  .style("stroke-linecap", "butt")
  .attr('x1', x1*this.scalingFactor) // x position of the first end of the line
  .attr('y1', yaxis) // y position of the first end of the line
  .attr('x2', x2*this.scalingFactor) // x position of the second end of the line
  .attr('y2', yaxis);}

To fill color in those blocks I created pickColorn function

 pickColorn(feat:any) {
pattern.generate(['#1f77b4'])
let colors_feature:any = { UTR5:'red', CDS:'green', Intron:'blue', UTR3: 'orange'}
  return colors_feature[feat]}

How could I create a matrix grid layout with lines for a view?

How could I create a view that has a matrix grid layout starting from the center and extending to the bottom. Im not allowed to post pictures yet but it would look like this ->

Matrix Graph Layout

Everything I have looked up shows me fancy grids. But I am looking for something more simple, kind of like this

            Rectangle()
                .fill(AppColor.purpleBackGroundLines)
                .frame(width: .infinity , height: 1, alignment: .center)

            Rectangle()
                .fill(AppColor.purpleBackGroundLines)
                .frame(width: .infinity , height: 1)
                .padding(.top)

but by adding lots of rectangles and paddings, it keeps pushing my center rectangle higher and I want it to have a fixed center position. I also would like to customize how much padding there is in between each line with no pushing, and if it can be absolute in every screen no matter the dimension. Is there an efficient way of doing this? Thank you.

C# class pattern sending messages back up chain

this is probably a fairly noob question - I'm not sure what the correct way to perform the below is, this occurs quite often in my code and there must be a "proper" solution / pattern to solve this. Consider the following:-

    class First
        {
            Second second = new Second();
            public void talkToSomethingExternal()
            {
                second.secondEvent(); // At some point I've decided I want to fire an event down the chain
            }
            public void sendMessage(string message)
            {
    
            }
    
    
        }
    
        class Second
        {
            Third third = new Third();
            public void secondEvent()
            {
                third.thirdEvent();
            }
        }
    
        class Third
        {
            public void thirdEvent()
            {
                // I want the first class to send a message, sendMessage("Message")
            }
        }

The method I normally end up taking is as below, passing a reference to the top class:-

 class First
    {
        Second second;

        public First ()
        {
            second = new Second(this);
        }
        public void talkToSomethingExternal()
        {
            second.secondEvent(); // At some point i've decided i want to fire an event down the chain
        }
        public void sendMessage(string message)
        {

        }


    }

    class Second
    {
        Third third; 

        public Second(First first)
        {
            third = new Third(first);
        }
        public void secondEvent()
        {
            third.thirdEvent();
        }
    }

    class Third
    {
        First storeFirst;

        public Third(First first)
        {
            storeFirst = first;
        }
        public void thirdEvent()
        {
            storeFirst.sendMessage("Message");
        }
    }

I could also just use a return type from the event, however again the messages are being passed from class to class back up the "chain" and also if any return value is required from sendMessage(..) this would be difficult and even more convoluted to achieve.

It seems a messy way to do this. I am also not aware of seeing this behaviour in the various frameworks (take WinForms as an example), so is there a different way of approaching this? I'm also finding it difficult to ascertain what this problem is actually called.

Thanks

Python "socket" library - where is real implementation? [duplicate]

When i searched implementation of listen method in _socket.pyi i founded only this code with ellipsis:

class socket:
    # ... some another methods with ellipsis ...
    def listen(self, __backlog: int = ...) -> None: ...

I could not find a real implementation of listen(). What is this pattern when methods without explicit implementation are used in the main class of the module? What is the point of this approach?

samedi 22 octobre 2022

Generic method using instanceof and can it be avoided?

I have the following generic interface/classes

public interface AnInterface <T>{
    T convertToY();
    String getName();
    String getLastName();
}  

public class Foo implements AnInterface<Foo>{
   // some methods specific to Foo  
}   

public class Bar implements AnInterface<Bar>{
   // some methods specific to Bar
} 

I created a generic method as follows (I didn't think it would work but it does)

public static <T> List<Person> getPersons(AnInterface<T> in) {
        System.out.println(map.get(in));
        List<Person> list = new ArrayList<>();
        if (in instanceof Foo) {
            Person p = new Person(in.getName(), in.getLastName());
            p.setId(((Foo) in).getId());
            list.add(p);
        }
        else if(in instanceof Bar) {
            Person p = new Person(in.getName(), in.getLastName());
            p.setCC(((Bar) in).getCC());
            list.add(p);
        }
        return list;
    }  

The issue is that I did not create the classes so I am not sure if doing public class Foo implements AnInterface<Foo> is something usual or not.
So what I am interested in is if there is any issue with the generic method and using instanceof and if there is any problems that can be created. E.g. if I recall correctly we can't use instanceof on collections so I was wondering if I might make it easy to introduce some other bug with this approach.

Note:. I can't modify AnInterface or Foo or Bar

Removing code duplication and redundancy based on different criteria and priority level

I was wondering if I could get your suggestions on how to remove the code duplication/repetition here.

Any preferred and correct way of doing so?

Summary: This function returns a car's name and accepts the car code as a parameter

public string GetMatchingCar(string carCode)
{
    //CarResults is the list of cars that consists the details of the car as a collection.
    var carResults = new List<Car>
    {
        new Car{Name="AB car", Code="AB", Colors = new List<string>{"blue","green","black"}, Features=new List<string>{"compact","fast","light"} },
        new Car{Name="AC car", Code="AC", Colors = new List<string>{"gray","white","yellow"}, Features=new List<string>{"extended","fast","heavy"} },
        new Car{Name="DE car", Code="DE", Colors = new List<string>{"red","green","purple"}, Features=new List<string>{"sports","light"} },
        //and so on
    };

    //Specifications is a list of specification to choose from that includes color and feature.
    var specifications = new List<Specification>
    {
        new Specification{Color="blue", Feature="heavy"},
        new Specification{Color="red", Feature="light"},
        new Specification{Color="maroon", Feature="compact"},
        new Specification{Color="black", Feature="manual"},
        new Specification{Color="neon", Feature="heavy"},
        //and so on
    }

    //Now, we need to return the car based on the priority of criteria. Say P1, P2 and P3. P1's priority being highest.
    //P1: Code + Color + Feature
    //P2: Code + Feature
    //P3: Feature
    //just for example, priorities can be from P1 - P7

    //Now we have to combine the specification with carCode to check the priorities and return the matching car.

    //TODO: Here, is where I think I am doing wrong by having each for loop for each priority (as priorities can be upto Seven). All the foreach loop won't be executed if a priority is matched and returns a car.
    //TODO: Wondering how could I improve this?

    //priority P1: Code + Color + Feature 
    foreach(var specification in specifications){
        var matchingCarP1 = carResults.FirstOrDefault(x=> x.Code.Equals(carCode) && x.Colors.Contains(specification.Color) && x.Features.Contains(specification.Feature)); 
        if(matchingCarP1 != null) return matchingCarP1.Name;
    }

    //priority P2: Code + Feature
    foreach(var specification in specifications){
        var matchingCarP2 = carResults.FirstOrDefault(x=> x.Code.Equals(carCode) && x.Features.Contains(specification.Feature)); 
        if(matchingCarP2 != null) return matchingCarP2.Name;
    }

    //priority P3: Feature 
    foreach(var specification in specifications){
        var matchingCarP3 = carResults.FirstOrDefault(x.Features.Contains(specification.Feature)); 
        if(matchingCarP3 != null) return matchingCarP3.Name;
    }
    
    //Other priorities

    return string.Empty;
}

Any suggestions or feedback on this would be really helpful and highly appreciated! Thank you!

Design Patterns examples in real (open-source) projects

For a project, I'm looking for actual examples of design patterns in real projects or applications; for example, I found Observer and Singleton pattern in Elasticsearch. Does anyone know a well-documented project which uses design patterns and could provide examples in the code?

vendredi 21 octobre 2022

Is there a design pattern for the case of too many injected classes/dependencies?

I have a class that does too many things (many responsibilities). E.g. logging, loading data from various databases, methods that are doing extraneous calculation/processing as well as the main processing method that is this class's responsibility.
I have split the class to multiple classes each doing a specific thing and inject the classes to the original one.
The concern I have is that now I am injecting quite a few classes and I am not sure if that is considered bad design or code smell.
Is there some design pattern, similar as there is e.g. builder pattern for the case of having functions with too many parameters, that addresses the case of having many injected dependencies in a class?

jeudi 20 octobre 2022

How to properly return a list item for editing?

I created a class for convenient creation of a list that consists of different types of objects with a single interface.

A list item is created by calling a class method, the result of the method is a newly created object in the list for EDIT.

While this code works, I'm not sure if it's safe to return a list item to edit properties?

If not, how can you safely return an element from the list so that you can edit it.


var builder = new ElementBuilder();
var item = builder.AddText("abc");

// Edit new reference element property, should be stored in "builder" variable
item.Bold = true;

.....

// Pass builder instance with added items to other place

// If we do something like this with added values on above
var x = builder.First();
Console.WriteLine(x.Text);
Console.WriteLine(x.Bold);

// Should output:
abc
true
public class ElementBuilder
{

    private List<IElement> elements;

    public TextElement AddText (string text)
    {
        var item = new TextElement (text);
        elements.Add(item);
        return _elements.First(x => x == item);
    }

    public ImageElement AddImage (string imageUrl)
    {
        var item = new ImageElement (imageUrl);
        elements.Add(item);
        return _elements.First(x => x == item);
    }

    .....
}

mardi 18 octobre 2022

Java: Extend a static class from non static class

I have a class that is non-static. Let's say class A.

public class A

Can I extend this class to create an inner class?

Let's say

public class B {
    public static class nestedB extends A
    {
     // stuff
    }
}

Is this a good idea, or is this a good design?

Thanks!

Best practice for return statements in Python [closed]

I am wondering what are the best practices for return statements. In particular, is it ok to create an intermediate variable right before a return statement if I can otherwise bypass the variable and have the return statement followed by the value to return?

Example:

def add_me(a=1,b=2,c=3,d=4):
    e = a + b + c + d
    return e

or would it be better to do:

def add_me(a=1,b=2,c=3,d=4):
    return a + b + c + d

What about multiple function calls:

def sub_me=1,b=2,c=3,d=4):
    return a - b - c - d

def add_me(a=1,b=2,c=3,d=4):
    return a + b + c + d + sub_me() + 1

C# WinForms: FlowLayoutPanel FlowDirection issue

I'm trying to add x Buttons to a FlowLayoutPanel which is docked (DocType fill) in a Panel. I have set the FlowDirection to Topdown as I want something like:
this

However, the FlowLayoutPanel has a horizontal scrollbar, not a vertical one:
See here

I add the buttons like this:

flowLayoutPanel1.Controls.Add(new Button { Text = "Chat", Width = flowLayoutPanel1.Width - flowLayoutPanel1.Margin.Left - flowLayoutPanel1.Margin.Right});

I also tried:

flowLayoutPanel1.Controls.Add(new Button { Text = "Chat", Dock = DockStyle.Fill});

And also:

flowLayoutPanel1.Controls.Add(new Button { Text = "Chat", Anchor = AnchorStyles.Left | AnchorStyles.Right});

I tried this and also looked here. Both don't work for me.

Thanks for your help.

Artificial intelligence graphics designer

What if you can have a robot to do all the work for you and save you the stress of outsourcing for an expensive graphics designer.?? Couple with $500 dollars free Google ads credit to promote your business.

lundi 17 octobre 2022

Create a microservice for distributed cache (Redis)?

We have decided to introduce Redis to our microservice system as a distributed caching and in memory database. The initial decision is to create a service wrapping Redis, and a Client library will be created for other services to consume the Models.

So, a lot of models will be moved from other microservices to the "caching service". The service has web api for populate/update the data in Redis. The Client class library will export methods such as

ModelA GetModelA(...)
ModelB GetModelB(...)

And other services call the method above to get the data.

It seems there are two issues for this design?

  1. It seems the "Caching service" will be the sigle point of failure
  2. Other microservices are sharing the libary of the "caching service" project.

redesign to accomodate this use case

I have various python machine learning models/classes that have the following class defs:

class Model(pl.LightningModule):
   def __init__(self, **kwargs):
       # perform initialisation   

I am using pytorch lightning which also allows one to create a model instance from a saved checkpoint.

For each class I provide a load method as:

def load(**kwargs):
    if "checkpoint_file" in kwargs:
        # call the class method provided by the parent
        return Model.load_from_checkpoint(kwargs["checkpoint_file"])
    else:
        return MyModel(**kwargs)

I need to do this for every class as I need to provide a single callable instansiator (due to use of facebook hydra). I was wondering if there is some design pattern I can utilise to not have to define this for every class.

Design pattern to refactor lengthy task into a chain of unrelated classes

I currently have a long function that essentially carries out multiple different tasks sequentially (eg. it first clones a git repository, then edits some file in that repository, creates a new branch etc). What would be a clean way to refactor this code to a design pattern? I was thinking of creating a separate class for each sub class, and then chain these tasks together, but I am not sure if this is the best way of doing it. After reading more on the Chain of responsibility pattern, its principle is different to what i am trying to accomplish. My codebase is using python

Ideal outcome ->

MainClass calls classes
TaskA
TaskB
TaskC

dimanche 16 octobre 2022

Angular Should I create service for each entity?

In my angular application, I want to include the user token in each of my http request, so for what i have learned it is best to create a service so i dont have manually include my token or other parameter each time for my http request. The code could be like this:

export class FacilityService{
  constructor(private http: HttpClient) {
  }
  const httpOptions = {
    header: new HttpHeaders({
    Authorization: `bearer ` + JSON.parse(localStorage.GetItem('user')).token
    })
  }
  

  load(): Observable<any> {
    return this.http.get<any>(
      `${environment.rootUrl}/facility/getFacility`, httpOptions
    );
  }
}

However, I also learned to use the Service to store state. So we dont have to request from the server each time. The Code could like this:

export class FacilityService{
  facility: Facility[] = [];

  constructor(private http: HttpClient) {
  }
  const httpOptions = {
    header: new HttpHeaders({
    Authorization: `bearer ` + JSON.parse(localStorage.GetItem('user')).token
    })
  }
  

  load(): Observable<any> {
    if(this.facility.length > 0) return of(this.facility);
    return this.http.get<any>(
      `${environment.rootUrl}/facility/getFacility`, httpOptions
    ).pipe(map(facility => 
                 this.facility = facility;
                 return facility;
          ));
  }
}

2 question:

  1. if I suppose to create a local state for all HTTP requests of all my entities? Each time I check the local data first and if it does not exist I request from the server. And I will have facility service, user service, order service.

  2. How should I merge all those services into one service so I don't have to create the service for each of my entities? example, users data, and orders data can also share the service

Creating a plugin system in React/Node

TL;DR

I want to build a plugin/extension system for a React/Node.js app similar to what VS Code or Jira has. The plugins can be created by users and need to run in a sandbox environment inside of the main React app while having access to some API's provided by this main app. I did a lot of research and haven't been able to find useful information. I'm not necessarily looking for a complete solution or even a partial solution. Some insights, tips or any advice would be greatly appreciated if you have experience working with or building such architectures.

The problem

Note: I know my problem is probably overengineering a simple solution, but my goal is to build a robust and stupidly extensible ecosystem for a stupidly simple task (and learning in the process). It's not about finding the easiest and fastest solution, it's about sending a message.

So, my V1 application was a simple scoreboard controller for handball matches, it worked just fine in production for over a year now, but we've made some questionable design decisions early on and modifying a simple thing became a pain, let alone adding a whole new feature. That's why we've decided to start over from scratch and create a whole new app with more capabilities, including support for multiple sports. I know from the beginning that maintaining everything in a single codebase would lead to the same roadblocks we faced with V1. The first idea was to make a monorepo, where every sport was its own library extending the same core library and plugging into the main application. This would be fine, but I want to have more control over how the individual pieces connect and give the ability to other developers to extend the system (very unlikely that anyone would ever want to do it, but hey, the option would be nice).

Also, these plugins should be accessible from a "marketplace", kinda like Atlassian's apps. I've worked on a Jira extension at my job and really liked how it fit into their ecosystem. I've yet to try to "reverse-engineer" it or look into how Atlassian integrates these plugins into their system but since that's closed-source I don't think that would get me very far. I know in the frontend it's using an iframe to display the app inside Jira, which is a valid option I'm considering too.

One proposed solution overcomplication

Please skip to the App structure section to know more about the actual architecture. I know, I know, I should've started with that (or never written this but it's already too late)

My first idea was to have a few building blocks built into the app and have the game logic be stored in a JSON/YAML format. That would make storing the extensions in a database trivial and everything could be served from our backend. This file would contain definitions like what values a game has (scores, penalties, timeouts etc) what events are sent when some conditions are met, how the controllers and displays are constructed and where and how these values are connected. An interpreter would then extract the logic and run it on the corresponding locations: build the HTML template for the display and the controller and start the game with the timers and events on the backend magically connecting them. As you can see this is a very complicated way of solving the problem and the more I work on it the more I realize it's either going to become something like a programming language itself or I have to compromise on the extensibility which was the whole point in the first place. I'm already questioning my choices in life at this point.

What I'm dreaming of

I found this project react-pluggable the other day and it was everything I've had in mind. With one exception. Of course, the React modules have to be imported which defeats the whole purpose of disconnecting the plugins and storing them in a database or hosting them on a different server. I wish there was a solution to fetch components over the internet, dynamically, in runtime. And preferably cache them in case there is no internet connection which is apparently a required use case (more on that later).

App structure

I've ranted enough about what I want to achieve, now I want to give some context how the application is structured. The CLIENT is an Electron application running locally. It talks to a Node.js the server through ws but that's not always an option so it has to work independently and we shouldn't worry about it now. The CONTROLLER is a React app that will control the DISPLAY which is displayed by the CLIENT. The CONTROLLER sends events to the CLIENT like "hey, the home team scored a goal", and the CLIENT updates the game state and sends an event to DISPLAY like "hey, you should be displaying 1-0". Also, the clock complicates things because it has to be synced with everything. The CLIENT can also send events to the DISPLAY like "hey, this player's 2 minute penalty is over, you should update accordingly". So the CLIENT is always the source of truth in the system.

The information flows DISPLAY <- CLIENT <-> CONTROLLER. And this is where the whole plugin thing comes into the picture. The CLIENT doesn't know what to do except the basic functions like "start the clock", "update the clock", "wait until the timeout is finished" and "update the DISPLAY's DOM somewhere". and forwarding messages back and forth. The CONTROLLER doesn't know how to run a game, it only knows "there should be a button saying HOME +1 that sends a home goal event". And the DISPLAY is even more clueless, it only knows "monkey receive update to DOM, monkey updates the DOM".

All this magic in connecting the different parts of the application should be encapsulated in a single package (preferably, or in 3 submodules for the different parts) that could be fetched at runtime.

Final thoughts

Huh, that was a lot, I hope you're still with me. If you have any input on how this problem could be solved/simplified or have any ideas on how you would approach it, please share it with me. And if there's any more detail I could add or edits I could make to clarify, please let me know.

samedi 15 octobre 2022

Associate metadata tags with other tags and coding references

I have a series of tags that are to be associated to various other entities.

CREATE TABLE `CustomTags` (
  `id` int unsigned NOT NULL AUTO_INCREMENT,
  `tag_name` varchar(255) NOT NULL,
  PRIMARY KEY (`id`))
id tag_name
1 restaurants
2 sunsets
3 business meetings
4 nature
5 snow
6 friendly gatherings
7 winter sports

I have other tables linked with a N:N relationship with this table (i.e. there is an intermediate table that links the ids of each of the table for each relationship). When I load the data I need to be able to add a new tag:

id tag_name
8 winter sports lakes

and be able to:

  1. Load the new tag and associate it with an existing tag conditionally, e.g. if I have a set of other data linked with tag_id 7 then if new data are inserted that are associated with something similar, e.g. 8, I want to be able to group them all as one logical entity i.e. associate <7, winter sports> with <8, winter sports lakes> as in winter sports lakes is an extension of winter sports. And I would like to be able to do it conditionally, e.g. if in the end I want to remove anything associated to tag_id 8 I'd like to be able to do it without other impact.
  2. I'd like to be able in the code to be able to know when I am processing, e.g. restaurants vs business meeting. Originally in the code I used hard-coded id but this doesn't work well because the code is coupled with the ids in the database. The original issue I was trying to avoid is to use hard-coded strings in the code, e.g. restaurants or winter sports that could end up due to a typo to cause a problem in the data. I am not sure how to be able to have the code understand what kind of data it is processing when required.

How can I set my DB tables and definitions in the code to achieve this?

How to handle complex Aggregate Root invariant in DDD?

I have a library system that allows borrowing books for chosen time. I need to protect it from ordering the books if any of them is already fully booked in any point in this time.

So let's say in our stock we have these books and their counts:

Fellowship of the ring (x10)
The Shining (x1)

The user A decides to order both of these books in a quantity of 1 between 2022-01-01 and 2022-01-20.

When user B wants to order both of these books in quantity of 1 between 2022-01-15 and 2022-01-30 an error should be thrown, because The Shining is not available anymore.

Additionally - there is a configurable margin before and after each order to prepare (eg. 1 day would mean that user B can make the order starting as early as 2022-01-17).

Also at any point in time, library may buy new books and add them to stock, meaning that user B would be able to make an order.

Now the logic to check if books are available needs to find all orders for specified time, including the restocking margin and compare it to the overall available stock.

To simplify it I have split the time into 30min slots, so we can iterate through all slots in a given timeframe and calculate the stock for this slot.

Here is a sketch of my aggregate:

BookOrder

Members:
DateTime From
DateTime To
Book[] BooksOrdered

Methods:
CanOrder(from, to, books, stocksForTimeSpanWithMargin)
Order(from, to, books, stocksForTimeSpanWithMargin)

The problem is that all this logic is quite complex and the only way to enforce it via an aggregate root I can think of is retrieving ALL of the orders and feeding the aggregate with them via constructor. But it certainly wouldn't scale to 10000+ books, 10000+ orders per day for 2 years in future.

To reduce the data pushed into aggregate I could make the query for only specified items in only the specified time slots. It could still be a lot, but significantly less.

However this introduces another problem - part of the logic would be located in the DB query. How would aggregate root know if it received all orders from the timespan? It would have to work only with what it was given.

Error & exception handling design patterns, across distributed systems

I'm beginning an investigation into how we might handle errors across our many distributed services at work better.

Some services are very isolated, and others obviously not so much, the front end often would like to know about errors to display something predictable, these are all common requirements.

Right now errors are created per service/repo, and tracked/passed up very individually. This can cause its own problems I have discovered, trying to show an error on the front end has hand me creating three or more separate errors in 3 separate repos, passing them up. Each error in service has its own code style and even if i wanted to use the same they might be taken already etc, and the scope for mess is high.

Is there such a pattern or service where errors are, for instance centrally created, and then that service imported (or language specific clients/libraries generated) in each service where needed to share errors and their codes and validated front end error text for copy changes or more maybe.

I either envision a separate service (maybe even with a front end, that would allow an easy creation) this service could also scan our repos to give reports of error logs or exceptions that don't contain a property that links back to the error service, so we can slowly try and migrate over. This however sounds high in complexity.

Or maybe create a pub sub based service that collected and adds to error messages along its way, that sounds like a service that could be a single point of failure for all of this too, as opposed to a client/library import of a build.

I feel like this should have been solved before and i'm not doing a very good job of searching. I'm willing to follow a sensible pattern for distributed systems, or use off the shelf libraries or services that work across multiple languages.

I write this because we use datadog, and its clear when i'm writing monitors/alerts that we lack a coherent strategy across the board and its most obvious when im trying to write a monitor reading log output across multiple services, it would be nice if i could refer to something as a source of truth somewhere.

How to make polymorph registration in Laravel?

The design system provides three types of accounts: client, carrier, seller. Each one has own set of fields like name, direction, minprice.

Here is a register controller with register method.

public function register(Request $request) {}

Should I split that method on three methods by account type? Another way is create a three services and factory. The account type must have own validations rules, model etc.

Which way to choose to be flexible?

Is there a UML diagram describing TensorFlow implementation? [closed]

For my semestral project I'm searching for design patterns in the implementation of TensorFlow library. Is there some UML diagram of TensorFlow which can I use for my purpose?

Thanks :)

vendredi 14 octobre 2022

how convert api response to a certain font in laravel 8?

api response which I want to convert

//Fetching news in the API

public function newsFetchApi()
{

    $responseFromApi = Http::withHeaders([
        'X-API-KEY' => 'XXXXXX',

    ])->withBasicAuth(
        'XXXXXX',
        'XXXXXX'

    )->post('https://XXXXXX.in/filesystemapi/api/V1/Filesystemnews/');


    $response =   $responseFromApi['data']['news'];
    return $response;
}

I want to covert the news-title and news-content in a given font by client before it fetching the response

is it a good way to use del and delete dataframes at the end of jupyter notebook?

I was wondering, is it a good way to delete the data frames used in jupyter notebook at the end of it or more generally, delete variables when we are done with them(for freeing memory)? if it is a good pattern, when we should delete them? used special cell(s) for this purpose or what?

REST Api Routes format with filter

I am relatively new to the environment of professional use of API Routes, so a basic question.

Im working with VueJS asFrontend and node Express as Backend. How the GET Request against the api have to look like? I wonder how to deliver filter requests with a REST API?

Example:

GET https://example.com/api/v1/cities -> get all cities.

GET https://example.com/api/v1/cities?country={countrId} -> get all cities from this country with the id = {countrId}.

Question: Is this an API antipattern? If so, what would the route look like?

Note: A colleague told me it should look like this:

GET https://example.com/api/v1/cities?country.id={countrId}

SOLID IoC based on configuration in Spring Boot

I would like to have a collection of connections (REST, gRPC, RabbitMQ) for one or more different types of services. For example, with this configuration:

connections:
  http:
    - name: connectionA
      host: localhost
      port: 8081
      endpoints:
        - name: employees
          httpMethod: GET
          endpointType: EMPLOYEE
        - name: departments
          httpMethod: POST
          endpointType: DEPARTMENT
  grpc:
    - name: connectionB
      host: localhost
      port: 8084
      disableTls: true
      endpoints:
        - name: employees
          endpointType: EMPLOYEE

When using the EmployeeService I would have an EmployeeHttpClient and an EmployeeGrpcClient, as well as a DepartmentHttpClient in the DepartmentService. Those clients would have the concrete implementation using the specific connection details, but the service shouldn't care about that.

What do you think about this approach?

What would be the correct way of doing it in Spring Boot?

jeudi 13 octobre 2022

Node js concurrent DB I/O operations in mongoDB makes node memory frozen or fully occupied for certain process

NodeJS concurrent DB I/O operations in mongoDB makes memory frozen or fully occupied for one process. I'm not able read the DB queries when more concurrent write/read happens in the mentioned architecture which i've attached here in drive link. The memory frozen happens on any of the 3 services when the high load occurs.

https://drive.google.com/file/d/18Wkdv_gRzeW7LKrd-CdvTAyzjMLxtP5s/view?usp=sharing

Design pattern of a circular buffer using external memory C++ [closed]

I'm working in an implementation of a circular buffer in an embedded system (microcontroller) using external memory chips for storing data logging. The circular buffer itself is not stored in internal ram, just the current node or data to be written.

I have different memory types: flash, magnetic... which work very different internally. To expand life span, flash will be written only when a full sector can be filled, but with others as soon as there is data.

What is the best design pattern to use to abstract the logic behind a circular buffer and the logic needed for each memory type?

My objective is to be able to be able to implement same circular buffer independent of the underlaying memory type and use the same interface. I've been thinking using an abstract class buffer and then derive a new object for each memory type using the virtual calls when writting. Maybe I'm missing other better patterns.

SPA frontend/backend strategy and green IT [closed]

I'm wondering how frontend and backend should interact through http requests in a Single-Page-Application (SPA), considering green IT recommandations (such as limiting the number of http requests) and what consequences it would have on the way I write my backend controllers. I'm looking for a pattern to use in any case (if possible)

Basically, I have a Page in frontend that will need multiple data from backend (let's say: the page's content and some featured news).

What would be the best choice? (considering green IT, but also performance, dev patterns, etc)

  1. Having an action in the controller for each request (2 actions here). So, the frontend would need 2 http requests to retrieve data.
  2. Having one and only one action in the controller for both request. So, the frontend would need only 1 (but heavier?) http request to retrieve data.

And is this choice the one to use on the whole project (for a Page that needs 100 different data)? Or should I need to use the other pattern depending on the case (in which cases?) ?

Best practices for routing based on App State in flutter (navigator 2.0)

I am stuck on the Architecture design of my App. Assuming I have a Tristate (Error, Loading, Data), what is best practice in modern flutter (go_router, riverpod/provider, ...) for bringing this to life?

Two options:

  • One page (so no routing) and complex widgets, one shown for each tristate. Statemanagement based on e.g. provider which triggers a rebuild when the state changes.
  • Three pages (which allows for reuse based on e.g. different errors, data, etc) seems cleaner in design BUT now the router needs to decide based on what is happening in the Tristate

How can the three page way be implemented? A provider is not the way since this leads to all state being above the router, always. Further the loading page for example would need to navigate in the build method, which is only possible with hacks.

So while the three page design seems more clean to me (loading needs to know nothing about error needs to know nothing about data presentation) I feel like the implementation will not be clean in anyway.

How can the three page way be implemented? Or why should it not be? (Does anyone have some tips, pointers, articles or a best practice?)

What is Recommended design pattern for Flutter Web [closed]

  1. I want to learn about design pattern for Flutter Web App. Which design pattern is good for flutter web application.

Filtering for multi level nested checkboxes in Django

We are in the early stages of development and we are looking for an approach to build an API using DRF to support filtering when there are multiple groups

The Filter looks as follows:

There are few group that goes to 3 levels, few to 2 level and few doesn't have any level at all. The data filtering should work when selecting any of the check boxes and the API should return the filtered response

I am beginner in Django and so far has worked on basic filtering with one level like Purchase.objects.filter(purchaser=user)

Any guidance on designing the model and API structure would be much appreciated

mercredi 12 octobre 2022

What does really mean Single Responsibility

I am learning SOLID principles. I read a lot of things about the single responsibility principle but I don't get it exactly. I will give an example that what do I want to say.

Let's say we have an article service for managing articles. How should I design article service by single responsibility.

Is like that:

class ArticleService{

  create(){};  
  read(){};
  update(){};
  delete(){};

}

Or create a class for each operations like that :

class ArticleCreateService{

  create(){};

}

class ArticleReadService{

  read(){};

}

// and so on ...

According to single responsibility which is the best way to managing articles?

Thanks.

Builder Pattern python: 1 builder or 2 builders inheriting from base?

I've a need to use a builder for constructing an entity (let's call it MessageWrapper) which needs different processing before initialization, according the type of entity that was passed to the builder.

Currently I'm using one builder. Users of the Builder will be able to pass it (currently) two types of entities which conform to an "interface".

def set_entity(self, entity: IMyInterface):
   self._entity = entity
   return self

# IMyInterface is ABC with abstract process() method
def _process_entity_field(interface_instance: IMyInterface):
    ...
    return interface_instance.process()

def build(self):
    ...
    final_message_wrapper_field = self._process_entity_field()
    ...

Here's where my build() method is getting a bit messy though: MessageWrapper also requires another field processing_type: str, which will be either "A" or "B" (but in the future, may be "C").

I'm already taking care of the processing stage by using IMyInterface.process(), but I find myself doing this in the build() method:

if isinstance(self._entity, AThatConformsToMyInterface):
    processing_type = "A"
elif isinstance(self._entity, BThatConformsToMyInterface):
    processing_type = "B"
else:
    raise RuntimeError(
        f"Entity of type {self._entity.__class__.__name__} is not supported"
    )

Which kinda smells?

The alternative is to eschew the interface, and break out the builder to two builders that inherit from a common base:

a) MessageWrapperFromAObjBuilder - which will simply accept AThatConformsToMyInterface and simply set processing_type to "A".

b) MessageWrapperFromBObjBuilder - which will simply accept BThatConformsToMyInterface and simply set processing_type to "B".

which removes the need for the isinstance()... check, because we know which one of the XThatConformsToMyInterface we got.

The disadvantages of this alternative as I see them:

  1. In my mind the builder pattern is there exactly for cases where you need some kind of conditional processing in the build stage. So using two builders seems kinda meh.

  2. If type CThatConformsToMyInterface is introduced later, I'll probably have to have a designated builder for that.

So, one builder, or two builders? (it's worth emphasizing: the builder is building the same type of object at all times, it's how they get built/processed and their processing_type: str value that depends on XThatConformsToMyInterface.

To make things worse/more-challenging:

  • BThatConformsToMyInterface is an external library class.
  • I don't want to introduce another method to the IMyInterface "interface" which hints at what processing_type: str should be.

Which design pattern is best suited for an Upgrade/Buff system?

I am creating a game (C#/Unity) and am trying to implement an Upgrade/Buff system. This project is also a platform for me to learn and implement different design patterns. I want to the system to work as follows:

-The buff contains all needed operations within itself and can be allied/removed easily.

-The buff can be a simple stat increase, but also have more complex behaviour, like applying damage over time in an area of effect.

The design pattern that seems most useable to me right now is the Decorator pattern. Maybe also the visitor pattern.

Does anybody with experience with such a system have some thougths about this?

Thanks!

mardi 11 octobre 2022

RegEx pattern - length 7, first character letter and second numeric third character letter remaining numeric [duplicate]

I am trying to create a RegEx to match a string with the following criterion

  • Length 7
  • First character is a letter, second one is a number and 3rd character is a letter and afterwards the remaining 4 characters remaining numbers.

Examples

A0M8997
B0M9998
g8j8989

I have tried this code. but it's not working

/[A-Za-z]{1}[0-9]{1}[A-Za-z]{1}[0-9]/

Is there a best practice to call a specific function every time before any function in Dart/Flutter?

I am working with singleton pattern, and I have managed the dependency and parameter injections like this:

  LocalizationClientComponent._();

  /// Singleton instance of the [LocalizationClientComponent] class.
  static final instance = LocalizationClientComponent._();

  static bool _injectedDependencies = false;

  static bool _injectedParams = false;

  late final DeviceInfoRepository _deviceInfoRepository;
  late final MapapiRepository _mapapiRepository;
  late final SensorsRepository _sensorsRepository;
  late final LocationRepository _locationRepository;
  late final BluetoothRepository _bluetoothRepository;
  late final WifiRepository _wifiRepository;

  void injectDependencies({
    required DeviceInfoRepository deviceInfoRepository,
    required MapapiRepository mapapiRepository,
    required SensorsRepository sensorsRepository,
    required LocationRepository locationRepository,
    required BluetoothRepository bluetoothRepository,
    required WifiRepository wifiRepository,
  }) {
    if (_injectedDependencies) return;

    _deviceInfoRepository = deviceInfoRepository;
    _mapapiRepository = mapapiRepository;
    _sensorsRepository = sensorsRepository;
    _locationRepository = locationRepository;
    _bluetoothRepository = bluetoothRepository;
    _wifiRepository = wifiRepository;

    _injectedDependencies = true;
  }

Then the same thing for parameters:

  void injectParams({
    required String email,
    required String password,
    required Duration postFrequency,
    required int sensorListLength,
  }) {
    if (_injectedParams) return;

    _postFrequency = postFrequency;
    _sensorListLength = sensorListLength;
    _telemetryEvent = TelemetryEvent(_sensorListLength);

    _injectedParams = true;
  }

After that, I made this method to control whether params and dependencies are injected:

void _handleInitError() {
    final StringBuffer errorMessage = StringBuffer();
    if (!_injectedDependencies) {
      errorMessage
        ..write(
          'Dependencies are not injected. Use injectDependencies() before using any other methods.',
        )
        ..write('\n');
    }

    if (!_injectedParams) {
      errorMessage.write(
        'Parameters are not injected. Use injectParams() before using any other methods.',
      );
    }

    if (errorMessage.isNotEmpty) {
      throw Exception(errorMessage.toString());
    }
  }

Now, I am adding _handleInitError() to all other methods like this:

Future<bool> openAppSettings() async {
    _handleInitError();

    return _locationRepository.openAppSettings();
  }

  Future<bool> openLocationSettings() async {
    _handleInitError();

    return _locationRepository.openLocationSettings();
  }

  Future<bool> requestLocationPermission() {
    _handleInitError();

    return _locationRepository.requestLocationPermission();
  }

  Future<bool> activateLocationService() async {
    _handleInitError();

    return _locationRepository.activateLocationService();
  }

  Future<bool> turnOnWifi() async {
    _handleInitError();

    return _wifiRepository.turnOnWifi();
  }

Is there a better way than adding all methods to _handleInitError()?

Is there a way to automatically execute this method before any method?

lundi 10 octobre 2022

C# Mediatr Request Handler Injected with multiple Repositories

Im using MediatR in .Net Core and kind of confused if injecting multiple repositories to handle a business logics is an acceptable/clean way of doing it?

My sample Code:

public class MyRequestHandler: IRequestHandler<...>
{
  public IHeaderRepository _headerRepository;
  public IChildRepository _childRepository;
  ///constructor dependency injection happening here

  public async Task<...>Handle(.....request,..... cancellationToken)
  {
     var header = await _headerRepository.GetHeader(..headerId);
     if(header != null) await _headerRepository.Insert(...):

     await _childRepository.Insert(..., ...headerId)
  }
}

How to pass the set[State] function to a non-descendent trigger component

Here is the diagram. ChildComponentB has a state - stateX. In ChildComponentA, once the event occurs, it will change the stateX in ChildComponentB.

If the ChildComponentB is the child component of ChildComponentA, then it's easy, just pass the setStateX as a prop to ChildComponentA. But in this case, it's not.

a brief description diagram

The real scenario is the following. I have a canvas component, there are some static Rectangles already there, once there are mouse move over the line of the Rectangles, I'd like to add the indicator lines to another child component of the canvas component.

Hence, the rectComponent is not the descendent of the distanceIndicatorsComponent. So I can't pass the setLines to RectComponent.

What's your approach to do that?

enter image description here

Tipps for defining an architecture to abstract over and unify different CRUD APIs

I am currently trying to build a library to abstract over the different CRUD Web APIs of our business units. I cant give an actual example, but here is the base problem:

We have N different APIs that all share more or less the same resources. There is for example a resource to create a user. For one API X, the resource is called "user", for another API Y its "users" and they all require different fields for a Create Request. X requires only a name and an address as string while Y wants name as a string but address as an object consisting of street, city and country.

My first intention was to use the Adapter pattern and define my own class User along with the API specific interfaces.

interface XUser {
   name: string;
   address: string;

   CREATE() : XUserWithId
}

interface YUser {
   name: string;
   address: {
      street: string;
      city: string;
      country: string
   };
   
   CREATE() : YUserWithId
}

interface IUser {
   name: string;
   street: string;
   city: string;
   country: string;
}

class User implements IUser {}

Then I wanted to write my Adapters to make my User class behave like the API specific objects:

class XUserAdapter implements XUser {
   constructor(user: IUser) {
      this.name = user.name;
      this.address = [user.street, user.city, user.country].join(", ");
   }
}


class YUserAdapter implements YUser {
   constructor(user: IUser){
      this.name = user.name;
      this.address = {
         street: user.street;
         city: user.city;
         country: user.country;
      }
   }
}

CREATEing a User Resource in APIs X and Y now boils down to creating one user object, putting it into the adapters and calling the adapters CREATE functions:

users: User[] = [];

tom = new User(name="Tom", street="Foostreet", city="Bartown", country="USA");

xuser = new XUserAdapter(tom);
yuser = new YUserAdapter(tom);

xuser = xuser.CREATE();
yuser = yuser.CREATE();

My Problem is that obviously calling CREATE will hand me back platform specific user objects that i want to be in the shape of my own User class. This would require two more adapters to "translate" to the opposite direction:

class UserFromYUser implements IUser {
   constructor(yUser: YUser) {
      ...
   }
}

Question: Am I missing the obvious? Is creating one adapter per platform specific resource really the way to go? I Read about the two-way adapter pattern in GoF but it was just a short hint. Do you guys have any other idea on how one could tackle this problem and how to split the code but maintain consistency across API interfaces?

Thank you in advance.

How to handle 2 event source using 2 separate thread in golang

This is a design-related question. I have a situation where my application receives events from 2 different sources it is registered with and the application should handle events from these 2 sources parallelly. My Application is already handling events from one source using the buffered channel (where events are queued up and processed one after another). Now I am in a situation where the application needs to handle the events from a different source and I cannot use the same channel here because the Application may have to handle events from these 2 sources parallelly. I am thinking of using another buffered channel to handle the events from the second event source. But I am concerned about the same resource being used to process 2 events parallelly. Even though we use channel we need to again apply sync while processing these events.

Could you please suggest me a better way, any patterns I can use, or a design to handle this situation?

This is the code I have now to handle event from one source

for event := range thisObj.channel {

    log.Printf("Found a new event '%s' to process at the state %s", event, thisObj.currentState)

    stateins := thisObj.statesMap[thisObj.currentState]
    // This is separate go routine. Hence acuire the lock before calling a state to process the event.
    thisObj.mu.Lock()
    stateins.ProcessState(event.EventType, event.EventData)
    thisObj.mu.Unlock()

}

Here thisObj.channel is being created at start up and the events are being added in a separate method. Currently this method is reading events from the channel and processing the events.

dimanche 9 octobre 2022

Good structure/ design pattern for scraping multiple pages?

in my new project I want to scrape & aggregate different data from multiple websites.

We are talking about a number of specific, previously known websites, each of which can provide up to N attributes. Comparable to a product on different online stores.The input is always the link and then you should get the scraped data as result. I have already implemented the scraping of the data and it works for the different websites. So it is only about the structure.

Now for my questions:

  1. Do you have any tips on how to structure the basic "framework" in the most sustainable way to keep it low-maintenance and extensible? My first thought was to have the different scrapers inherit in OOP style from a general abstract scraper, so they can all run together in the main-script. However, I am not quite sure about this yet. Would appreciate any ideas concerning the design?
  2. Since some pages give less info, I thought to save all findable information into one big dict. Does this idea make sense or should I define an separate class?

Additional information (if important):

  • it's only about the structure and the main-script, the scraping in detail is already working.
  • There is not much data (n<500) involved, so I don't expect to get problems with IP bans. However, I would like to design it as "smart" as possible.
  • For scraping I use BS4, Requests, Selenium and AJAX.
  • it is primarily about targeted data retrieval, no complicated crawling

Thanks in advance!