@Matt Clarke, @Timothy Spann, @Shishir Saxena, @Bryan Bende, @milind pandit, @Dan Chaffelson, @Pierre Villard, @Andrew Grande Apache NiFi is installed on non-hadoop environment and targets to ingest processed files into HDFS (Kerberized cluster). Is it workable solution to achieve above use case as I face multiple error even after performing below activities. Please advise if there is anything additional I have to perform. * Firewall restriction between NiFi and management server is open and ports (22,88,749,389) are open. * Firewall restriction between NiFi and edge node server is open and ports (22, 2181,9083) are open * krb5.conf file from hadoop cluster along with keytab for application user is copied to NiFi server. Running kinit using application user and keytab - successful token is listed under klist. * SSH operation is successful and also SFTP into hadoop server works fine. * configured hdfs-site.xml and core-site.xml files into NiFi. * PutHDFS processor fails to ingest data throwing authentication error. Is there something I am missing here. Hadoop environment : 4 management node and 1 edge node on public network within cluster and 4 worker nodes on private network. Alternative solution to this, I have installed NiFi in edge node and everything works fine. I need to make Apache NiFi work on non-hadoop environment and ingest data into HDFS and Hive. ... View more

Hi. I have a configuration to fetch tweets via getTwitter filtered by language and set of keywords (or conditions and also 2 word combination - eg: nifi,spark,scala,spark dataframe,spark rdd, spark dataset). I have a similar combination of 210 words and I see the results with sometimes non-matching tweets and also duplicates. Please advice. @Scott Shaw @Artem Ervits @Timothy Spann @Pierre Villard ... View more

Please advise on resolving this connectivity issue. NiFi installed on non-hadoop environment as standalone application is accessing hive on a clustered hadoop environment. What should my Hive metastore URI be : I tried with multiple options of values from hive-site.xml and also replacing that with edge node ip address. Facing the below error. Suspect its because of IP restrictions from non hadoop server to access hadoop edge node or Hive metastore URI as per hive-site.xml. Please advice. ERROR [Timer-Driven Process Thread-9] o.a.n.processors.hive.PutHiveStreaming PutHiveStreaming[id=87b35ed3-015b-1000-8c8d-a99b97d59afa] PutHiveStreaming[id=87b35ed3-015b-1000-8c8d-a99b97d59afa] failed to process session due to com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: Unable to instantiate org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: Unable to instantiate org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient ERROR [Timer-Driven Process Thread-9] o.a.n.processors.hive.PutHiveStreaming com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: Unable to instantiate org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection timed out (Connection timed out) Support from you guys is much appreciated - @ Andrew Grande, @Timothy Spann, @Constantin Stanca, @Matt Burgess ... View more