Created 11-30-2020 04:25 AM
I have 20 CSV files and I need to convert these files into parquet format before loading them to S3. I prepared schemas for 20 files(different schema for each file). For this to work I need to route all these files using route on attribute and process. This works but it create flow with 20 convert record processors. I want to know is there a way that I can store all these 20 schemas and I can use them according to filename with single convert record processor.
Thanks.
Created 11-30-2020 05:02 AM
Want to get a detailed solution you have to login/registered on the community
Register/LoginCreated 12-01-2020 08:28 AM
Want to get a detailed solution you have to login/registered on the community
Register/LoginCreated 11-30-2020 05:02 AM
Want to get a detailed solution you have to login/registered on the community
Register/LoginCreated 12-01-2020 06:44 AM
@stevenmatisonIn the link that you provided, they explained how to setup HWX schema registry. But I don't have access for HWX schema registry. So I wanted to go ahead with AvroSchemaRegistry controller service. Could you please show how to add schemas to AvroSchemaRegistry and use them dynamically according to filename.
Thanks.
Created 12-01-2020 08:28 AM
Want to get a detailed solution you have to login/registered on the community
Register/Login