NodeJS Worker Pattern with external plugins

I am writing a NodeJS service that aggregates data from many different sources. The sources it pulls data from are in the form of Plugin classes, and each Plugin has one or more DataSource objects that handle data fetching and processing.

I am trying to implement the Worker Pattern, where the service will load each plugin and its respective DataSource classes and spawn a pool of workers.

Implementing this would be trivial if all of the plugins did the same exact thing, but it becomes more difficult when considering the post-processing operations that each of the individual plugins define.

Consider the following pseudocode for an example DataSource:

class DataSource {
  GetWork() {
    LookForNewData()
    LookForExistingUnprocessedData()
  }

  DoWork( work ) {
    DownloadNewData()
    ProcessNewData()
  }
}

It would be one thing if the DataSource simply downloaded data and saved it directly to a database, but since each DataSource processes the downloaded data differently, the worker needs to be able to have the class and its processing functions readily available.

I have considered a few solutions, but none of them seem very appealing:

• Have a worker process for each DataSource
  • This could ultimately result in hundreds of child processes
• Pass the work and the location of the DataSource.js file to the worker, have it include and cache the instance
  • This is the most promising solution, but seems very fragile and unnecesary

Are there any design patterns or existing solutions to a problem like this that can offer a cleaner implementation of what I am trying to do?