@Predrag Minovic Thanks for the answer. impressive. I was assuming the root node (catalog) needs to be mentioned in the xmlinput.start and xmlinput.end, so that all the nodes in between the root nodes can be queried using the xpath. Thanks for the clarification. ... View more

Pls share the ambari logs (if you are using ambari for the deployment), else share the yum install logs. If a forward proxy infront of the server blocks the connection to remote public repository, installation using local repository should work fine. you could check if you have http_proxy variable set (if yes, try disabling and again try with the local repo.).; ... View more

LD_LIBRARY_PATH=:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.4.0.0-169/hadoop/lib/native ls -ltr /etc/shadow -r-xr-x--- 1 root hive 1809 Jun 28 09:20 /etc/shadow Error: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000/default: Peer indicated failure: Error validating the login (state=08S01,code=0) java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000/default: Peer indicated failure: Error validating the login at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:210) at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:156) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:208) at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:142) at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:207) at org.apache.hive.beeline.Commands.connect(Commands.java:1149) at org.apache.hive.beeline.Commands.connect(Commands.java:1070) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:52) at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:980) at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:705) at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:767) at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:485) at org.apache.hive.beeline.BeeLine.main(BeeLine.java:468) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: org.apache.thrift.transport.TTransportException: Peer indicated failure: Error validating the login at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:199) at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:307) at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:185) ... 24 more Beeline version 1.2.1000.2.4.0.0-169 by Apache Hive @Benjamin Leonhardi thanks for the inputs. Sorry I had to recreate the cluster for some reasons, it took some time on this. yes, the user is already part of the OS (username and password are valid). Still beeline returns the same error message. Also the user home directory exists in the hdfs filesystem - ... View more

Hi Neeraj, I have tried this approach on a xml which does not have an attritbute value.. but the query returns a null value. can you pls guide https://community.hortonworks.com/questions/40979/hive-xml-parising-null-value-returned.html. ... View more

job runs fine when i turned off the node manager on the node1. Thanks. I will search on how to run based on the node labels. ... View more

Thanks, Pradeep. now the error msg differs. Resource crunch on the node1 seems to the cause. I will try disabling the node1 node manager and re-run the job. 16/05/25 13:24:20 INFO mapreduce.Job: Running job: job_1464166918626_0020 16/05/25 13:25:06 INFO mapreduce.Job: Job job_1464166918626_0020 running in uber mode : false 16/05/25 13:25:06 INFO mapreduce.Job: map 100% reduce 0% 16/05/25 13:25:12 INFO mapreduce.Job: Job job_1464166918626_0020 failed with state KILLED due to: MAP capability required is more than the supported max container capability in the cluster. Killing the Job. mapResourceRequest: <memory:2048, vCores:1> maxContainerCapability:<memory:1024, vCores:3> Job received Kill while in RUNNING state. 16/05/25 13:25:12 INFO mapreduce.Job: Counters: 6 Job Counters Killed map tasks=1 Total time spent by all maps in occupied slots (ms)=0 Total time spent by all reduces in occupied slots (ms)=0 Total time spent by all map tasks (ms)=0 Total vcore-seconds taken by all map tasks=0 Total megabyte-seconds taken by all map tasks=0 16/05/25 13:25:12 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead 16/05/25 13:25:12 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 68.7027 seconds (0 bytes/sec) 16/05/25 13:25:12 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 16/05/25 13:25:12 INFO mapreduce.ImportJobBase: Retrieved 0 records. 16/05/25 13:25:12 ERROR tool.ImportTool: Error during import: Import job failed! ... View more

Hi @Benjamin Leonhardi. thanks for this fantastic article. I just got it implemented. a) Created a new OS user account/password (user1/user1), then tried authentication from beeline, it got failed b) Tried for an another OS user account/password(hive/hive), it got authenticated for the password set on the OS level. so password validation happens against the pam module. but the same didn't happen for user1/user1. Please let me know, do we need to create the same user account in metastore database as well ? if yes, i'm bit confused. beeline> !connect jdbc:hive2://node2.cluster.net:10000/hive Connecting to jdbc:hive2://node2.cluster.net:10000/hive Enter username for jdbc:hive2://node2.cluster.net:10000/hive: hive Enter password for jdbc:hive2://node2.cluster.net:10000/hive: **** Connected to: Apache Hive (version 1.2.1000.2.4.0.0-169) Driver: Hive JDBC (version 1.2.1000.2.4.0.0-169) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://node2.cluster.net:10000/hive> [root@node1 ~]# [root@node1 ~]# beeline WARNING: Use "yarn jar" to launch YARN applications. Beeline version 1.2.1000.2.4.0.0-169 by Apache Hive beeline> !connect jdbc:hive2://node2.cluster.net:10000/hive Connecting to jdbc:hive2://node2.cluster.net:10000/hive Enter username for jdbc:hive2://node2.cluster.net:10000/hive: user1 Enter password for jdbc:hive2://node2.cluster.net:10000/hive: ***** Error: Could not open client transport with JDBC Uri: jdbc:hive2://node2.cluster.net:10000/hive: Peer indicated failure: Error validating the login (state=08S01,code=0) 0: ... View more

Thanks @neeraj overlooked the yum errors in the logs. I created a local repository with Oracle 6.4 media but that resulted in lot of error as most of the previous packges in the system was updated using the latest Oracle Public Repository so I have manually installed fuse and all other rpms demanded by Deployement wizard. And then started the deployment. It went well.. Cluster is now up and running. ... View more