@Foivos A The banner is a NiFi core feature and is not tied in anyway to the dataflows you select or build on your canvas. You are correct that the best approach for identifying which dataflows on a single canvas are designated dev, test, or production is through the use of "labels". In a secure NiFi setup, you can use NiFi granular multi-tenancy user authorization to control what components a user can interact with an view. If you use labels, you should set a policy allowing all user to view that specific component, so even if they are not authorized to access the labeled components, they will be able to see why via the label text. Thanks, Matt ... View more

Thank you, it's the batch schedule that needed to be changed in my case. ... View more

@Bryan Bende thank you, I'm trying with SplitContent, initial results are encouraging. ... View more

thanks @Matt Clarke. Will downgrade asap. ... View more

@Raj B Its true that it won't change the layout, however, you can create an external table on top of the same location in its original format i.e. create external table abc (col1 int, col2 string) location '/location/of/modified/orc/table' stored as textfile (or original format); Ideally if the data has not changed then you should be able to view it and then read or do a CTAS from that. ... View more

@Bryan Bende, placed the phoenix jar file in NiFi's work directory path and restarted NiFi instance; now seeing a different error in the log when I enable the HBase client service - "org.apache.nifi.StdErr java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z" Failed to invoke @OnEnabled method due to java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z 2017-05-10 14:13:50,016 ERROR [NiFi logging handler] org.apache.nifi.StdErr [StandardProcessScheduler Thread-6] ERROR org.apache.nifi.controller.service.StandardControllerServiceNode - 2017-05-10 14:13:50,016 ERROR [NiFi logging handler] org.apache.nifi.StdErr java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z 2017-05-10 14:13:50,016 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.hadoop.security.UserGroupInformation.<init>(UserGroupInformation.java:623) 2017-05-10 14:13:50,017 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1200) 2017-05-10 14:13:50,017 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.nifi.hadoop.SecurityUtil.loginKerberos(SecurityUtil.java:52) 2017-05-10 14:13:50,017 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.nifi.hbase.HBase_1_1_2_ClientService.createConnection(HBase_1_1_2_ClientService.java:226) 2017-05-10 14:13:50,017 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.nifi.hbase.HBase_1_1_2_ClientService.onEnabled(HBase_1_1_2_ClientService.java:178) 2017-05-10 14:13:50,017 ERROR [NiFi logging handler] org.apache.nifi.StdErr at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2017-05-10 14:13:50,017 ERROR [NiFi logging handler] org.apache.nifi.StdErr at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2017-05-10 14:13:50,017 ERROR [NiFi logging handler] org.apache.nifi.StdErr at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2017-05-10 14:13:50,017 ERROR [NiFi logging handler] org.apache.nifi.StdErr at java.lang.reflect.Method.invoke(Method.java:498) 2017-05-10 14:13:50,018 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137) 2017-05-10 14:13:50,018 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125) 2017-05-10 14:13:50,018 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70) 2017-05-10 14:13:50,018 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47) 2017-05-10 14:13:50,018 ERROR [NiFi logging handler] org.apache.nifi.StdErr at org.apache.nifi.controller.service.StandardControllerServiceNode$2.run(StandardControllerServiceNode.java:348) 2017-05-10 14:13:50,018 ERROR [NiFi logging handler] org.apache.nifi.StdErr at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 2017-05-10 14:13:50,018 ERROR [NiFi logging handler] org.apache.nifi.StdErr at java.util.concurrent.FutureTask.run(FutureTask.java:266) 2017-05-10 14:13:50,018 ERROR [NiFi logging handler] org.apache.nifi.StdErr at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) 2017-05-10 14:13:50,019 ERROR [NiFi logging handler] org.apache.nifi.StdErr at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) 2017-05-10 14:13:50,020 ERROR [NiFi logging handler] org.apache.nifi.StdErr at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 2017-05-10 14:13:50,020 ERROR [NiFi logging handler] org.apache.nifi.StdErr at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 2017-05-10 14:13:50,020 ERROR [NiFi logging handler] org.apache.nifi.StdErr at java.lang.Thread.run(Thread.java:745) 2017-05-10 14:13:50,020 ERROR [NiFi logging handler] org.apache.nifi.StdErr [StandardProcessScheduler Thread-6] ERROR org.apache.nifi.controller.service.StandardControllerServiceNode - Failed to invoke @OnEnabled method of HBase_1_1_2_ClientService[id=102e119a-19a2-1409-671f-dddd93a063de] due to java.lang.NoSuchMethodError: org.apache.hadoop.security.authentication.util.KerberosUtil.hasKerberosKeyTab(Ljavax/security/auth/Subject;)Z ... View more

@Raj B The PutHDFS will "close" the file after each append. The writing of a file every minute will be less efficient than say writing a bigger file every ten minutes, but that is something you will have to determine which is more important to your use case. But even just waiting a minute is still more efficient then writing 100 files a sec. ... View more