I am using NiFi for my data flow and then I kick off ETL script which runs many (hive/Pig) MR/Tez jobs. Is there easy way to detect (ie trigger) once the job has finished. Creating a trigger manually per job is not scalable since this are many jobs. Going into each job and have it create a trigger is off the table. ... View more

Documentation for PutHiveStreaming requires flow file to be in avro format. My understanding is hive streaming only support ORC format. When puthivestreaming is used, does it convert avro to orc prior to inserting into hive table? trying to understand the functionality. ... View more

The documentation for ListFile states: If the primary node changes, the new Primary Node will pick up where the previous node left off without duplicating all of the data. How does the "new" primary node pick up where the previous node left off without flow file duplication? I ask since the previous primary node may have the file flow, when new primary node is elected how does it primary node get the flow file without duplicating or cloning it? ... View more

I have cluster which is NOT kerberized. Is it possible to enable user impersonation for hive queries run from zeppelin? All the documentation seems to require user princple. ... View more