How to make code compatible with different library version to be able to run on different environments?

I am working on a SQL-tool to extract and display data from the Hadoop ecosystem. I have used Spark and Hive libraries to achieve SQL connectivity and browsing.

I am using some APIs, classes, that I have extended from Spark and Hive and its internal methods that I have used. Now many of the APIs and Class constructors and method syntax have changed.

I need to some way to handle multiple version of Spark and Hive libraries in my code so that I can support wide variety to versions of Spark and Hive and its combinations.

In my tool, I had to extend many classes of Spark SQL and write my own implementations. To support multiple version of Spark I have currently used reflection to achieve Spark method/APIs call compatibility.

Example: Say ThriftBinaryCLIServive is a class that I have extended. The constructor has changed in Hive1.2 and Hive2.1. In Hive1.2, ThriftBinaryCLIServive(CLIService cliService) In Hive2.1, ThriftBinaryCLIServive(CLIService cliService, Runnable oomHook)

To solve this I have written a separate version of extended classes say: CustomThriftBinaryCLIServive1_2(compatible with Hive 1.2) and CustomThriftBinaryCLIServive2_1(compatible with Hive 2.1) and keep them different source folders. I use Maven to compile packages in specific source folders at a time. I use reflection to instantiate appropriate class on runtime depending on Hive version And use the ThriftBinaryCLIServive as the reference class.

Currently, this approach works for me. But I see code maintainability problems in future and code duplication to some extent and I am not sure if this is the right way to do it.

I have also tried a Custom Class Loader approach where I used two different class loaders to load class of different versions but did not help much, also that is not so manageable in long run and hard to debug as well.

Can someone suggest other better ways by which I can solve this problem?