CSV, JSON, XML, TXT mapping and conversion? Recommended framework/language?

I’m after opinions on the best framework/languages to use for a project at my work.

Basically, we need to operate as an integration platform for many customers (1000s). Each customer provides event data to share in their own format/content. (eg. CSV, JSON, XML, TXT, EDI).

All customers need to share data between each other - provided they have an agreement.

DataHub

I see the main challenge areas being:

1. Access rights to data and roles of each user
2. Creation of Mappings between the various formats (1000+ mappings)
3. Frequency and volume of:
   • Inserts
   • Exports

While I understand that all modern frameworks/languages are most likely up to the task, I’d like to consider optimal solutions. I have the luxury of a clean slate to work with – I am not tied to any technologies or frameworks.

Initially I thought Rails or Django would do the trick, however given the volumes of data mappings and subsequent runs of these, I might best consider a more performant solution?

What is the best language for parsing different data formats and converting to different formats? Files will not be huge, generally less than 1,000 events.

Will Ruby/Python be up to the task or am I better looking at different tech?

The UI will need to somewhat reactive/real-time indicating job progress to users with charting etc. to be added in the future.

I’d also like to build a “mapper” similar to the one offered by Dell Boomi to aid in the creation of both input and output mappings, which can then be run at user request.

Data Mapper

So, any thoughts and/or suggestions? I’d not be developing this myself, so need to consider tech for which I can hire resources.

Possible inclusions might be:

• List item

• Ruby on Rails

• React

• Svelte

• Django

• Go

• .Net Core

• Elixir-Phoenix

• Node (Framework?)

• Rust

• AWS API Gateway/Lambda

Cheers in Advance