mardi 7 juillet 2020

Better approach to call external API in apache beam

I have 2 approaches to initialize the HttpClient in order to make an API call from a ParDo in Apache Beam.

Approach 1:

Initialise the HttpClient object in the StartBundle and close the HttpClient in FinishBundle. The code is as follows:

public class ProcessNewIncomingRequest extends DoFn<String, KV<String, String>> {
        
        @StartBundle
        public void startBundle() {
            HttpClient client = HttpClient.newHttpClient();
            HttpRequest request = HttpRequest.newBuilder()
                                       .uri(URI.create(<Custom_URL>))
                                       .build();

        }
        
        @ProcessElement
        public void processElement(){
            // Use the client and do an external API call
        }

        @FinishBundle
        public void finishBundle(){
             httpClient.close();
        }
}

Approach 2:

Have a separate Class where all the connections are managed using the connection pool.

public class ExternalConnection{

       HttpClient client = HttpClient.newHttpClient();
       HttpRequest request = HttpRequest.newBuilder()
                                       .uri(URI.create(<Custom_URL>))
                                       .build();
       
       public Response getResponse(){
             // use the client, send request and get response 
       }
       
}

public class ProcessNewIncomingRequest extends DoFn<String, KV<String, String>> {
        
        @ProcessElement
        public void processElement(){
             Response response = new ExternalConnection().getResponse();
        }
}

Which one of the above 2 approaches are better in terms of performance and coding design standards?

Aucun commentaire:

Enregistrer un commentaire