I am working with context params now since most all sources are saying they are a better choice than variables.  However I'm not finding how I can set the context param values from anywhere other than the UI?  I need to be able to pass env values from my docker-compose file and ultimately have them end up as the value of a context param.  (This worked for variables.)  Context params don't seem to support EL so I can't pull outside values into the context param that way.  Any ideas?      ... View more

Hi, just starting off with NiFi. I am impressed with the orchestration capabilities and am now considering how to best handle the "T" in ETL. Ultimately I need to support hundreds of different formats of structured data, typically transforming into one or two target formats. Often these formats have hundreds of fields and significant munging is required to reach the target format. External enrichment might be required (NER, geocoding, you name it.) My thought right now is that I'll need a NiFi processor that handles translations using some external capability. Either my team can build this definition-driven translation engine and make it available to NiFi via a processor.... OR, has anyone here ever successfully integrated some other ETL tool? Open source and even commercial seems to have extraction and loading covered but I've not yet run into a great tool that specializes in allowing a data guru to create deep schema transformation definitions that can be executed at run time. The idea is roughly like XSLT in raw spirit but more more extensive in transformation capabilities.... would love to hear your recommendations if you have any. ... View more