Member since
03-18-2018
33
Posts
0
Kudos Received
0
Solutions
08-30-2018
12:51 PM
Thanks .Yes we do have postgres DB which used for ambari and Hive metastore
... View more
08-30-2018
12:44 PM
All nodes on our cluster require outage for maintenance . I am planning to stop all services in Ambari and stop Ambari server Are these steps correct & enough?
... View more
Labels:
05-29-2018
02:19 PM
We are getting below error on randomly for few minutes
and then goes away, its coming in PUThiveql 2018-05-29 01:01:07,279 INFO [Timer-Driven Process
Thread-95] org.apache.hive.jdbc.HiveConnection Will try to open client
transport with JDBC Uri:
jdbc:hive2://hdpmaster2.msoit.com:10000/default;principal=hive/_HOST@IT.COM 2018-05-29 01:01:07,281 ERROR [Timer-Driven Process
Thread-95] o.apache.thrift.transport.TSaslTransport SASL negotiation failure javax.security.sasl.SaslException: GSS initiate failed at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) at
org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) at
org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271) at
org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656) at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) at
org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:204) at
org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:176) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at
org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38) at
org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582) at
org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1148) at
org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106) at
org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044) at
org.apache.nifi.dbcp.hive.HiveConnectionPool.lambda$getConnection$0(HiveConnectionPool.java:355) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656) at
org.apache.nifi.dbcp.hive.HiveConnectionPool.getConnection(HiveConnectionPool.java:355) at
sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at
java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:89) at
com.sun.proxy.$Proxy97.getConnection(Unknown Source) at
org.apache.nifi.processors.hive.PutHiveQL.lambda$new$1(PutHiveQL.java:191) at
org.apache.nifi.processor.util.pattern.Put.onTrigger(Put.java:96) at
org.apache.nifi.processors.hive.PutHiveQL.lambda$onTrigger$6(PutHiveQL.java:274) at
org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114) at
org.apache.nifi.processor.util.pattern.RollbackOnFailure.onTrigger(RollbackOnFailure.java:184) at
org.apache.nifi.processors.hive.PutHiveQL.onTrigger(PutHiveQL.java:274) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1147) at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:175) at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at
java.lang.Thread.run(Thread.java:748) Caused by: org.ietf.jgss.GSSException: No valid credentials
provided (Mechanism level: Failed to find any Kerberos tgt) at
sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147) at
sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122) at
sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187) at
sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224) at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache NiFi
05-23-2018
04:39 PM
None of the above worked. It working after adding below configuration set("spark.jars","/usr/hdp/current/spark2-client/jars/sqljdbc42.jar")
... View more
05-23-2018
12:08 PM
Thanks Sridhar Already tried the --jars and spark.driver.extraClassPath & spark.executor.extraClassPath, they don't work. What do you mean by "Directory expansion does not work with --jars"
... View more
05-23-2018
01:03 AM
spark-submit --master yarn --deploy-mode cluster sqlserver.py --jars sqljdbc42.jar I get Error : java.lang.ClassNotFoundException: com.microsoft.sqlserver.jdbc.SQLServerDriver Everything works when i use --deploy-mode client and copy the jar file to /usr/hdp/current/spark2-client/jars/sqljdbc42.jar Should i copy the sqljdbc42.jar to /usr/hdp/current/hadoop-yarn-client/lib/ on all data nodes?
... View more
Labels:
- Labels:
-
Apache Spark
-
Apache YARN
05-03-2018
01:09 AM
Any update , its 2018. I am gettiing the same error, Error: Error while processing statement: FAILED: Hive Internal Error: org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAccessControlException(Query with transform clause is disallowed in current configuration.) (state=08S01,code=12)
... View more
05-02-2018
07:18 PM
I am trying to use python udf in hive, but i get the below error Error: Error while processing statement: FAILED: Hive Internal Error: org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAccessControlException(Query with transform clause is disallowed in current configuration.) (state=08S01,code=12)
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Ranger
04-14-2018
02:54 PM
I am also facing the same issue, did you find a resolution?
... View more
04-04-2018
04:53 PM
This is resolved.
Because hive is not configured with impersonation, table creation uses hive
user to create tables and landing directories in HDFS. added hive in
Ranger policy
... View more