lundi 27 avril 2020

Correct design pattern and technology to map functions to each other

Abstract:

I'm attempting to create a "data interoperability API" or in other terms "high-level query interface API" that will be consumed by (data scientists, web apps, any who wants to query multiple datasets).

Assumptions:

The underlying data will usually be in these formats:

1) Best case scenario - XML (with proper XSD). • the XML serves as meta-data that describes (where that data resides, file, web service,etc... field descriptions) • points to delimited data (CSV) or even binary data

OR

2) Just plain XML as meta-data (NO XSD).

• the XML serves as meta-data that describes (where that data resides, file, web service,etc... field descriptions) • points to delimited data (CSV) or even binary data

OR

3) Plain data (CSV no headers)

This API will be provided as distributable (In case of Java a JAR, or Python extension) that can be loaded by consumer applications.

Progress so far:

1) Im able to load XSD using JAXB for Java and PyXB for Python version and generate class based on XSD info. Im calling “xjc” as system process via Java :

   ProcessBuilder processBuilder = new ProcessBuilder(CMD_ARRAY)Process process = processBuilder.start();

2) Im able to also bind objects & data in memory and issue (what I call “native queries”).

 //1. instance 
            jaxbContext = JAXBContext.newInstance(clazz);
            //2. Use JAXBContext instance to create the Unmarshaller.
            unmarshaller = jaxbContext.createUnmarshaller();
            //3. Use the Unmarshaller to unmarshal the XML document to get an instance of JAXBElement.
            inStream = new FileInputStream( this.xmlFile);
            OaisInteropFrameworkStates.setState(OaisInteropFrameworkStates.STATE_LOAD_NATIVE_API);
            returnedObject  = unmarshaller.unmarshal(inStream);

3) Next step (High level Public API mapping) - What I need advice on , see the diagram below (Everything black boxes are tested and working refer to the red shapes).

High level data flow architecture:

enter image description here

Actual question 1) What is the best design pattern approach, in order to automatically map/wrap different functions?

Assumptions:

1) (User native to public mapping) - The user MUST provide a XML (or maybe JSON) 1 to 1 mapping of function signatures

2) The mapped function will always return a String

3) Native API must be decoupled from Public API and they have no knowledge of each other. (except via the mapping file or any auto-generated code based on the xml mapping file. Similar to client & server interfaces.

4) I have very limited time to work on this project, 8-10 hours per week. Therefore simplicity over elegance/

Some thoughts:

I was looking at XML-RPC Python (https://docs.python.org/3/library/xmlrpc.client.html#module-xmlrpc.client) or Java (https://www.tutorialspoint.com/xml-rpc/xml_rpc_examples.htm).

2) Also REST-RPC, or some other service-oriented architecture.

3) Have the user create XSD that maps the function endpoints and use JAXB (Rather not, XSD is too verbose, and these days Python minded scripting people will hate that...)

I like the approach were XML-RPC uses a similar mapping xml:

Request:

<?xml version="1.0" encoding="ISO-8859-1"?>
<methodCall>
   <methodName>sample.sum</methodName>
   <params>
      <param>
         <value><int>17</int></value>
      </param>

      <param>
         <value><int>13</int></value>
      </param>
   </params>
</methodCall>

Response:

<?xml version="1.0" encoding="ISO-8859-1"?>
<methodResponse>
   <params>
      <param>
         <value><int>30</int></value>
      </param>
   </params>
</methodResponse>

Thanks a million!

Aucun commentaire:

Enregistrer un commentaire