lundi 22 février 2021

Spring batch selecting item reader data provider based on specified source

I have a spring batch job that gets executed based on a HTTP request.

First step generates request file locally based on some static file (which contains list of instruments) in the classpath.

Second step uploads the file to a server.

@Configuration
@EnableBatchProcessing
public class BatchConfiguration {

    private final JobBuilderFactory jobBuilderFactory;
    private final StepBuilderFactory stepBuilderFactory;

    public BatchConfiguration(final JobBuilderFactory jobBuilderFactory, final StepBuilderFactory stepBuilderFactory) {
        this.jobBuilderFactory = jobBuilderFactory;
        this.stepBuilderFactory = stepBuilderFactory;
    }

    @Bean
    public Job generateRequest(final Step generateRequestStep, final Step uploadRequestStep) {
        return this.jobBuilderFactory.get("MyRequestJob")
                .start(generateRequestStep)
                .next(uploadRequestStep)
                .build();
    }

    @Bean
    public Step generateRequestStep(@Qualifier("instrumentFlatFileItemReader") final ItemReader<Instrument> instrumentItemReader,
                                    @Qualifier("") ItemWriter<Instrument> instrumentItemWriter) {
        return stepBuilderFactory.get("requestStep")
                .chunk(5)
                .reader(instrumentItemReader)
                .writer(instrumentItemWriter)
                .build();
    }


    @Bean
    @Scope(value = "step", proxyMode = ScopedProxyMode.INTERFACES)
    public FlatFileItemReader<Instrument> instrumentFlatFileItemReader(@Value("#{jobExecutionContext['feedType']}") String feedType) {
        FlatFileItemReader<Instrument> reader = new FlatFileItemReader<>();
        reader.setResource(new ClassPathResource("/path/to/static/file", getClass().getClassLoader()));

        DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer();
        tokenizer.setNames("column1", "column2");
        tokenizer.setStrict(false);

        DefaultLineMapper<Instrument> lineMapper = new DefaultLineMapper<>();
        lineMapper.setLineTokenizer(tokenizer);
        lineMapper.setFieldSetMapper(new InstrumentFieldMapper());
        lineMapper.afterPropertiesSet();

        reader.setLineMapper(lineMapper);

        return reader;
    }


    @Bean
    @StepScope
    public ItemStreamWriter<Instrument> instrumentItemStreamWriter(@Value("#{jobExecutionContext['FILE_OUTPUT_PATH']}") String requestFilePath) {
        BeanWrapperFieldExtractor<Instrument> fieldExtractor = new BeanWrapperFieldExtractor<>();

        DelimitedLineAggregator<Instrument> lineAggregator = new DelimitedLineAggregator<>();
        lineAggregator.setFieldExtractor(fieldExtractor);

        return new FlatFileItemWriterBuilder<Instrument>()
                .name("instrumentWriter")
                .resource(new FileSystemResource(requestFilePath))
                .delimited()
                .fieldExtractor(fieldExtractor)
                .lineAggregator(lineAggregator)
                .shouldDeleteIfExists(true)
                .build();

    }
}

currently, there is only one way to load the instruments, from the static file. I now have requirements where users can specify which media to load static from, e.g. local classpath, database or from call to webservice.

I am unsure of a pattern to follow to design this. Currently, I am using flat file item reader, but this won't work for database, external service. I need some help to put this behind an interface and have ability to derive the appropriate one based of the media type chosen:

for example: the http request can provide which media type to use:

{
  jobName: "REQUEST",
  cobDate: "2021-02-22",
  type: EOD,
  media: "DATABASE"
} //load list from database table

or

{
  jobName: "REQUEST",
  cobDate: "2021-02-22",
  type: EOD,
  media: "FILE_SYSTEM"
}  //load list from static file

Aucun commentaire:

Enregistrer un commentaire