Member since
10-17-2016
45
Posts
10
Kudos Received
3
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
795 | 04-21-2017 06:14 PM | |
2804 | 04-19-2017 05:59 PM | |
712 | 04-11-2017 08:24 PM |
04-11-2018
03:34 PM
@vishal dutt - Validate your hive section : Advanced hive-atlas-application.properties in Ambari
atlas.hook.hive.synchronous - boolean, true to run the hook synchronously. default false. Recommended to be set to false to avoid delays in hive query completion.
... View more
06-23-2017
07:10 PM
@Matt Clarke, @Timothy Spann, @Shishir Saxena, @Bryan Bende, @milind pandit, @Dan Chaffelson, @Pierre Villard, @Andrew Grande Apache NiFi is installed on non-hadoop environment and targets to ingest processed files into HDFS (Kerberized cluster). Is it workable solution to achieve above use case as I face multiple error even after performing below activities. Please advise if there is anything additional I have to perform. * Firewall restriction between NiFi and management server is open and ports (22,88,749,389) are open. * Firewall restriction between NiFi and edge node server is open and ports (22, 2181,9083) are open * krb5.conf file from hadoop cluster along with keytab for application user is copied to NiFi server. Running kinit using application user and keytab - successful token is listed under klist. * SSH operation is successful and also SFTP into hadoop server works fine. * configured hdfs-site.xml and core-site.xml files into NiFi. * PutHDFS processor fails to ingest data throwing authentication error. Is there something I am missing here. Hadoop environment : 4 management node and 1 edge node on public network within cluster and 4 worker nodes on private network. Alternative solution to this, I have installed NiFi in edge node and everything works fine. I need to make Apache NiFi work on non-hadoop environment and ingest data into HDFS and Hive.
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache NiFi
06-05-2017
07:34 PM
Hi. I have a configuration to fetch tweets via getTwitter filtered by language and set of keywords (or conditions and also 2 word combination - eg: nifi,spark,scala,spark dataframe,spark rdd, spark dataset). I have a similar combination of 210 words and I see the results with sometimes non-matching tweets and also duplicates. Please advice. @Scott Shaw @Artem Ervits @Timothy Spann @Pierre Villard
... View more
Labels:
- Labels:
-
Apache NiFi
04-25-2017
08:29 PM
Support from you guys is much appreciated - @ Andrew Grande, @Timothy Spann, @Constantin Stanca, @Matt Burgess
... View more
04-24-2017
07:47 PM
1 Kudo
Please advise on resolving this connectivity issue. NiFi installed on non-hadoop environment as standalone application is accessing hive on a clustered hadoop environment. What should my Hive metastore URI be : I tried with multiple options of values from hive-site.xml and also replacing that with edge node ip address. Facing the below error. Suspect its because of IP restrictions from non hadoop server to access hadoop edge node or Hive metastore URI as per hive-site.xml. Please advice. ERROR [Timer-Driven Process Thread-9] o.a.n.processors.hive.PutHiveStreaming PutHiveStreaming[id=87b35ed3-015b-1000-8c8d-a99b97d59afa] PutHiveStreaming[id=87b35ed3-015b-1000-8c8d-a99b97d59afa] failed to process session due to com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: Unable to instantiate org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: Unable to instantiate org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient ERROR [Timer-Driven Process Thread-9] o.a.n.processors.hive.PutHiveStreaming
com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: Unable to instantiate org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection timed out (Connection timed out) Support from you guys is much appreciated - @ Andrew Grande, @Timothy Spann, @Constantin Stanca, @Matt Burgess
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache NiFi
04-21-2017
06:15 PM
Deployed latest versions and able to access required hive processors.
... View more
04-21-2017
06:14 PM
Closed issue with data error.
... View more
04-21-2017
06:01 PM
1 Kudo
I am still facing this issue - Any support for resolution. How do I revert back and ensure my local ports are used for my sandbox. Describing the issue again - I was implementing the azure cloud deployment of HDP sandbox and faced above error, later when I had my local setup of sandbox on a virtual box - then I am unable to access 127.0.0.1 and respective ambari or zeppelin ports - Please advise as I need my local sandbox to work and ignore the azure stuff.
... View more
04-19-2017
05:59 PM
1 Kudo
Thanks @Timothy Spann. Looked at your comments just now after i did some modifications to existing nar file. I edited the current ScanAttribute file to iterate the dictionary words and check for its presence in twitter message. Customizing is good and look forward to make more solutions integrated into NiFi.
... View more