Member since
01-09-2016
70
Posts
30
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
7679 | 01-27-2016 04:47 PM |
07-07-2016
07:21 PM
Is there any way where I can migrate the running (scheduled state) of 100s feed/process on Falcon to another cluster? Like copying falcon store, embedmq, and/or graph database.
... View more
Labels:
- Labels:
-
Apache Falcon
05-06-2016
06:06 PM
@Jonas Straub This seems to be a security risk on hdfs. Any user wihtout having sudo su can become superuser by: export HADOOP_USER_NAME=hdfs
... View more
04-21-2016
10:10 PM
@Benjamin Leonhardi I setup the Hiverserver2 authentication with PAM using all 1-5 steps but unfortunately still getting invalid login error while using beeline. I used AMD64 JPAM lib. My hiveserver2 java lib path is: -Djava.library.path=:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.3.4.0-3485/hadoop/lib/native In HDP 2.3, there is no path: /usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64 I copied the .so in /usr/hdp/2.3.4.0-3485/hadoop/lib/native.
... View more
04-15-2016
05:07 PM
I have the following : hive-shims-0.20S-1.2.1.2.3.4.0-3485.jar hive-shims-0.23-1.2.1.2.3.4.0-3485.jar hive-shims-1.2.1.2.3.4.0-3485.jar
... View more
04-04-2016
08:21 PM
1 Kudo
HDP 2.3 ------------------ I am getting this error intermittently. Sometime the Hive action in oozie works fine. Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], main() threw exception, org.apache.hadoop.hive.metastore.IMetaStoreClient.isLocalMetaStore()Z
java.lang.NoSuchMethodError: org.apache.hadoop.hive.metastore.IMetaStoreClient.isLocalMetaStore()Z
at org.apache.hadoop.hive.ql.session.SessionState.unCacheDataNucleusClassLoaders(SessionState.java:1474)
at org.apache.hadoop.hive.ql.session.SessionState.close(SessionState.java:1468)
at org.apache.hadoop.hive.cli.CliSessionState.close(CliSessionState.java:66)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:683)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:306)
at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:290)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)
at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:241)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Oozie Launcher failed, finishing Hadoop job gracefully
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Oozie
02-27-2016
05:01 AM
@Junichi Oda @Ali Bajwa @spolavarap Did you got the solution?. I am struggling a lot and not able to search users within group. Here are my settings. Only groups getting fetched but no user. If I remove User Search Filter, I am able to fetch all users including users from other groups. Username Attribute = uid User Object Class = inetOrgPerson User Search Base = zz.com User Search Filter = (memberof=cn=TEAM_EDL_Dev,ou=Groups,o=zz.com) User Search Scope = sub User Group Name Attribute = memberof,ismemberof Group Member Attribute = member Group Name Attribute = cn Group Object Class = groupOfNames Group Search Base = zz.com Group Search Filter = (|(cn=edl*)(cn=TEAM_EDL_Dev)
... View more
02-08-2016
04:53 PM
It would be more helpful, if I should know on what reasons this errors comes, so that I can pin point the problem. Or please give me some steps to debug and solution.
... View more
02-07-2016
08:42 PM
1 Kudo
HDP-2.3.2.0-2950, Ambari 2.1, Hive: 1.2.1.2.3 I am facing problem in connecting beeline with secured ldap. Here are my testing, setups and errors: Secured LDAP Conenctivity Testing (working):
ping -c1 xxxx.net telnet xxxx.net 636 Setup: /etc/openldap/ldap.conf : TLS_CACERTDIR /usr/jdk64/jdk1.7.0_67/jre/lib/security Certifictae Type: CA certificate keytool -import -trustcacerts -alias xxxx -storepass changeit -noprompt -file 6a386909.0 -keystore /usr/jdk64/jdk1.7.0_67/jre/lib/security/cacerts (Certificate imported) Works Fine: ldapsearch -x -W -D 'uid=abc@xx.com,ou=People,o=xx.com' -H ldaps://xxxx.net:636 -b o=xx.com "(uid=abc@xx.com)" HDFS env Setup: export HADOOP_OPTS="-Djava_net_preferIPv4Stack=true
-Djavax.net.ssl.trustStore=/usr/jdk64/jdk1.7.0_67/jre/lib/security/cacerts
-Djavax.net.ssl.trustStorePassword=changeit ${HADOOP_OPTS}" Hive Advance Setup: hive.server2.authentication.ldap.baseDN : CN=%s,uid=%s,OU=People,O=xx.com
hive.server2.authentication.ldap.url : ldaps://xxxx.net beeline> !connect jdbc:hive2://<myhiveserver2-host>:10000 Connecting to jdbc:hive2://<myhiveserver2-host>:10000 Enter username for jdbc:hive2://<myhiveserver2-host>:10000: abc@xx.com (TRIED with just abc also) Enter password for jdbc:hive2://<myhiveserver2-host>:10000: ********* Error: Could not open client transport with JDBC Uri: jdbc:hive2://<myhiveserver2-host>:10000: Peer indicated failure: Error validating the login (state=08S01,code=0)
0: jdbc:hive2://<myhiveserver2-host>:100 (closed)>
hiveserver2.log ------------------
2016-02-07 20:06:07,764 ERROR [HiveServer2-Handler-Pool: Thread-47]: transport.TSaslTransport (TSaslTransport.java:open(315)) - SASL negotiation failure
javax.security.sasl.SaslException: Error validating the login [Caused by javax.security.sasl.AuthenticationException: Error validating LDAP user [Caused by javax.naming.AuthenticationException: [LDAP: error code 49 - Invalid Credentials]]]
at org.apache.hive.service.auth.PlainSaslServer.evaluateResponse(PlainSaslServer.java:109)
at org.apache.thrift.transport.TSaslTransport$SaslParticipant.evaluateChallengeOrResponse(TSaslTransport.java:539)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:283)
at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.security.sasl.AuthenticationException: Error validating LDAP user [Caused by javax.naming.AuthenticationException: [LDAP: error code 49 - Invalid Credentials]]
at org.apache.hive.service.auth.LdapAuthenticationProviderImpl.Authenticate(LdapAuthenticationProviderImpl.java:77)
at org.apache.hive.service.auth.PlainSaslHelper$PlainServerCallbackHandler.handle(PlainSaslHelper.java:106)
at org.apache.hive.service.auth.PlainSaslServer.evaluateResponse(PlainSaslServer.java:102)
... 8 more
... View more
Labels:
- Labels:
-
Apache Hive
02-05-2016
01:40 AM
1 Kudo
HDP 2.3, Ambari 2.1 Adding 'misfire_grace_time':10 to APS_CONFIG in /usr/lib/python2.6/site-packages/ambari_agent/AlertSchedulerHandler.py on every node and restarting ambari server and agent on all nodes didn't worked for me.
... View more
01-27-2016
06:11 PM
In HDP 2.3, Ambari 2.1, I did't find hive.server2.authentication.ldap.Domain
... View more