Member since
10-17-2016
11
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2716 | 02-23-2017 09:56 AM |
03-30-2017
11:15 AM
Hi All,We are using HDP hdp/2.4.2.0-258 version and sqoop version: sqoop-1.4.6.2.4.2.0-258 I am trying for sqoop incremental import, for that I need to connect to metastore to create sqoop jobs. But I couldnt find the sqoop-site.xml in /usr/hdp/current/sqoop-client/conf .. Can anyone point out the exact path.
... View more
Labels:
- Labels:
-
Apache Sqoop
03-10-2017
10:37 AM
It worked.. Thanks..
... View more
03-09-2017
06:48 AM
Hi All, I am trying to split a string with semicolon using the below query. In Beeline, Its thrwing me Parse execption error. But in Hive its working fine. I am properly escaping the semicolan in my query. Below is my query. select split(fieldname,'\;') from table name I also tried select split(fieldname,'\\;') from table name I also tried with double quotes. But Nothing is working in Beeline shell. But in Hive shell, no issues. Query is running fine. Beeline is throwing below error Error: Error while compiling statement: FAILED: ParseException line 1:29 cannot recognize input near '<EOF>' '<EOF>' '<EOF>' in select expression (state=42000,code=40000) Kindly let me know the proper way to escape the semi colans.
... View more
Labels:
- Labels:
-
Apache Hive
02-23-2017
09:56 AM
Adding the below Credential properties fixed my issue. <credentials>
<credential name="hbase-cred" type="hbase">
<property>
<name>hadoop.security.authentication</name>
<value>kerberos</value>
</property>
<property>
<name>hbase.security.authentication</name>
<value>kerberos</value>
</property>
<property>
<name>hbase.master.kerberos.principal</name>
<value>hbase/_HOST@XX.XX.COM</value>
</property>
<property>
<name>hbase.regionserver.kerberos.principal</name>
<value>hbase/_HOST@XX.XX.COM</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>${zookeeper_quorum}</value>
</property>
<property>
<name>hadoop.rpc.protection</name>
<value>authentication</value>
</property>
<property>
<name>hbase.rpc.protection</name>
<value>authentication</value>
</property>
<property>
<name>hbase.master.keytab.file</name>
<value>/etc/hadoop/hbase.keytab</value>
</property>
<property>
<name>hbase.regionserver.keytab.file</name>
<value>/etc/hadoop/hbase.keytab</value>
</property>
</credential>
</credentials>
... View more
02-22-2017
08:04 AM
@Ajay I am not an admin to do this. I believe sharelib is properly set. Hive Actions are running fine.
... View more
02-21-2017
05:49 PM
I'm running a test hbase java program via oozie java action. The following error is encountered : Error,
java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration
at org.apache.oozie.action.hadoop.HbaseCredentials.copyHbaseConfToJobConf(HbaseCredentials.java:67)
at org.apache.oozie.action.hadoop.HbaseCredentials.addtoJobConf(HbaseCredentials.java:55)
at org.apache.oozie.action.hadoop.JavaActionExecutor.setCredentialTokens(JavaActionExecutor.java:1287)
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1124)
at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1373)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:232)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
at org.apache.oozie.command.XCommand.call(XCommand.java:287)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:331)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:260)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745) The program runs correctly from command line : java -cp `hbase classpath` personal_project.HBaseConnector Is there a way I can pass output of 'hbase classpath' to the oozie java action. I have copied below hbase jars to workflow's lib directory as well.
hbase-common-$HBASE_VERSION.jar hbase-client-$HBASE_VERSION.jar hbase-server-$HBASE_VERSION.jar hbase-protocol-$HBASE_VERSION.jar hbase-hadoop2-compat-$HBASE_VERSION.jar htrace-core-*.jar hive-hbase-handler-$HBASE_VERSION.jar Following is the java action from workflow.xml : <workflow-app xmlns="uri:oozie:workflow:0.5" name="hbase_connect">
<global>
<job-tracker>${job_tracker}</job-tracker>
<name-node>${name_node}</name-node>
</global>
<credentials>
<credential name="hbase-cred" type="hbase">
<property>
<name>hadoop.security.authentication</name>
<value>kerberos</value>
</property>
<property>
<name>hbase.security.authentication</name>
<value>kerberos</value>
</property>
<property>
<name>hbase.master.kerberos.principal</name>
<value>hbase/_HOST@XX.XX.COM</value>
</property>
<property>
<name>hbase.regionserver.kerberos.principal</name>
<value>hbase/_HOST@XX.XX.COM</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>${zookeeper_quorum}</value>
</property>
<property>
<name>hadoop.rpc.protection</name>
<value>authentication</value>
</property>
<property>
<name>hbase.rpc.protection</name>
<value>authentication</value>
</property>
</credential>
</credentials>
<start to="hbase_connect_java" />
<action name="hbase_connect_java" cred="hbase-cred">
<java>
<job-xml>${runconfig_dir}/hbase-site.xml</job-xml>
<main-class>personal_project.HBaseConnector</main-class>
<java-opts>-Dsun.security.krb5.debug=true</java-opts>
<arg>${name_node}</arg>
<arg>${kerberos_principal}</arg>
<arg>${kerberos_keytab_file}</arg>
<arg>a:xxx,b:yyy,c:zzz,d:yyy</arg>
<arg>2016-02-22-123456</arg>
<file>${path_hdfs_common}/lib/hbase_connect.jar#hbase_connect.jar</file>
<file>${runconfig_dir}/jaas.conf#jaas.conf</file>
<file>${runconfig_dir}/krb5.conf#krb5.conf</file>
<file>${runconfig_dir}/hbase-site.xml#hbase-site.xml</file>
</java>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>${wf:name()} workflow failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
... View more
Labels:
- Labels:
-
Apache Oozie
12-01-2016
06:31 AM
@Pavel Benes Did you fixed the hive issues in Java Action from oozie ?
... View more
10-17-2016
07:27 AM
@Pavel Benes Did you get the solution. I am facing the similar issue. My java application writes the result to HDFS and it needs kerberos authentication. When schedule my application using oozie. I am facing this issue.
... View more