lundi 28 décembre 2015

java object designing help for ETL process

I am designing an ETL process where E ->Extractor : which is source and Transformer which transform content and Loader : is output where data to send/ store . I want all must be configurable means If currently source is file in future it can be database also same for output in future it can be database so class/object design should support this

I have following hierarchy of class 

public interface DataSource
{
public void init();
public Object getNextRecord();  
}

//I want to return generic result from getNextRecord 

public class FileSource implements DataSource
{
    public Object getNextRecord();
    {
        return file.readLine();     // one by one return file contains     
    }
}

public class SQLSource implements DataSource
{
    public Object getNextRecord();
    {
        Object[] result = new Object[colssize];
        for (int i = 0; i < cols; i++)
          result[i] = rs.getObject(i + 1);  

        return result: // return one row as object array        
    }
}

public interface Loader
{
    public void addRecord(Object object);
}

public class fileLoader
{
    public void addRecord(Object object)
    {
        writeLineToFile((String) object);
    }
}

public class SQLLoader implements Loader
{
    public void addRecord(Object object)
    {

// problem here getting object array and don't know the tables details writeObjectToDB(object) } } public class CapitalizedFileTransformer {

    @Override
    public void transform(DataSource source, Loader loader)
    {

    // for file source and file loader this works like this
        String data = null;
        while ((data = (String) source.getNextRecord()) != null)
        {
            loader.addRecord(data.toUpperCase());
            loader.addRecord("\n");
        }
        source.close();
        loader.close();


    // but for sql source and sql loader I have problem
    because its returning object array also don't have table info here to which I have to insert and don't want to change code of CapitalizedFileTransformer 
    }
}

It will be great if anyone can help me to re-design this classes I want a source and loader and transformer to be generic i.e any file source should work with sql loader and with any transformer

Aucun commentaire:

Enregistrer un commentaire