How to generate Avro Schema dynamically (during runtime) to validate a csv file?

@SashankRamaraju In most recent version of NiFi some of the older methods (infer schema) have been left behind.  You can certainly add them back in manually (PM if you want specifics).  However,  the current tools to manage record conversion are definitely preferred and bundle into NiFi out of the box on purpose.    

To solve your constantly changing csv, I would push back as to why the csv contents are changing.  If there was nothing I could do about it upstream, I would create a flow that split the different csvs up based on known schemas.  I would process the ones I have schema for and create a holder process group for those that fail.  I would monitor failures, create flow branches for new schemas (teaching my flow to be smarter over time).     After this kind of evaluation,  I would have a clear idea of how much the csv is actually changing.  I could now do some upstream actions on each csv to converge them into a single schema before I start processing them in Nifi.  For example, if some fields are missing,  I could do the work to add them (as empty values) before reading them with a single schema reader.  This gets a bit  kludgy but I wanted to explain the thought process and evaluation of how to converge the schemas into a single reader.   I would likely not do the later, and just split the flow for each csv difference.

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  

Thanks,

Steven @ DFHZ