About it_duanxiong

it_duanxiong · ‎10-10-2018

I want this function too.Current version do not have this function to finish this.

it_duanxiong · ‎09-17-2018

oozie can successed schedule the other mission , i.e. mapreduce?

it_duanxiong · ‎09-16-2018

I agree with you!you are right about this problem,First,have to try use spark-sumit to run this App,Then can use oozie go to schedule.

it_duanxiong · ‎09-14-2018

Just need to add this tag in wf.xml <property> <name>oozie.launcher.mapred.child.java.opts</name> <value>-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=8000</value> </property>

it_duanxiong · ‎09-13-2018

Hi,friend.I want to remote debug oozie workflow,but I don't know how to do it.I need some help.Thank you very much. I use oozie.sh jpda start,and use my IDEA connected,but i can't submit wf and the oozie process is on ,so somebody can tell me how to do next?

it_duanxiong · ‎09-06-2018

run：oozie admin -oozie http://localhost:11000/oozie -status have the question

it_duanxiong · ‎09-06-2018

wrong log: Error: IO_ERROR : java.io.IOException: Error while connecting Oozie server. No of retries = 1. Exception = Could not authenticate, Authentication failed, status: 302, message: Found oozie-site.xml: <property> <name>oozie.service.HadoopAccessorService.kerberos.enabled</name> <value>true</value> </property> <property> <name>local.realm</name> <value>HADOOP.COM</value> </property> <property> <name>oozie.service.HadoopAccessorService.keytab.file</name> <value>/usr/lib/cluster001/oozie-4.3.0/conf/oozie-http.keytab</value> </property> <property> <name>oozie.service.HadoopAccessorService.kerberos.principal</name> <value>oozie/ds1@HADOOP.COM</value> </property> <property> <name>oozie.authentication.type</name> <value>kerberos</value> </property> <property> <name>oozie.authentication.kerberos.principal</name> <value>HTTP/ds1@HADOOP.COM</value> </property> <property> <name>oozie.authentication.kerberos.name.rules</name> <value>DEFAULT</value> </property> keytab:(ds1上运行oozie-server，I have test this tap,It is successd) [root@ds1 conf]# klist -e -k -t oozie-http.keytab Keytab name: FILE:oozie-http.keytab KVNO Timestamp Principal ---- ----------------- -------------------------------------------------------- 1 09/05/18 14:56:16 oozie/ds1@HADOOP.COM (aes256-cts-hmac-sha1-96) 1 09/05/18 14:56:16 oozie/ds1@HADOOP.COM (aes128-cts-hmac-sha1-96) 2 09/05/18 14:56:16 HTTP/ds1@HADOOP.COM (aes256-cts-hmac-sha1-96) 2 09/05/18 14:56:16 HTTP/ds1@HADOOP.COM (aes128-cts-hmac-sha1-96) 1 09/06/18 19:43:58 oozie/ds1@HADOOP.COM (aes256-cts-hmac-sha1-96) 1 09/06/18 19:43:58 oozie/ds1@HADOOP.COM (aes128-cts-hmac-sha1-96) 2 09/06/18 19:43:58 HTTP/ds1@HADOOP.COM (aes256-cts-hmac-sha1-96) 2 09/06/18 19:43:58 HTTP/ds1@HADOOP.COM (aes128-cts-hmac-sha1-96)

it_duanxiong · ‎08-24-2018

This problem have to add spark.yarn.jars in sparkopts.and It have to start with"hdfs://"

it_duanxiong · ‎08-24-2018

It's my linux problem.

it_duanxiong · ‎08-23-2018

Hi,I have the same question,same error with one 1 action,I try your method,but it didn't work!

it_duanxiong · ‎08-20-2018

version: oozie:4.3 spark2.1 2018-08-20 19:33:23,998 [uber-SubtaskRunner] ERROR org.apache.spark.SparkContext - Error initializing SparkContext. java.lang.NullPointerException at scala.collection.mutable.ArrayOps$ofRef$.newBuilder$extension(ArrayOps.scala:190) at scala.collection.mutable.ArrayOps$ofRef.newBuilder(ArrayOps.scala:186) at scala.collection.TraversableLike$class.filterImpl(TraversableLike.scala:246) at scala.collection.TraversableLike$class.filter(TraversableLike.scala:259) at scala.collection.mutable.ArrayOps$ofRef.filter(ArrayOps.scala:186) at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:531) at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:527) at scala.collection.mutable.ArraySeq.foreach(ArraySeq.scala:74) at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:527) at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:868) at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:170)

it_duanxiong · ‎08-18-2018

Yaeh,I don't konw why,I change another cluster,every thing is fine.so thank you very much!

it_duanxiong · ‎08-18-2018

Thank you very much! It is a problem abour jar conflict.

it_duanxiong · ‎08-18-2018

When use spark2.1,We have to add oozie-sharelib-spark-4.2.0.jar in spark2

it_duanxiong · ‎08-17-2018

when I use oozie to schedule spark2 on yarn.I have e pproblem : Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, null java.lang.NullPointerException at scala.collection.mutable.ArrayOps$ofRef$.newBuilder$extension(ArrayOps.scala:190) at scala.collection.mutable.ArrayOps$ofRef.newBuilder(ArrayOps.scala:186) at scala.collection.TraversableLike$class.filterImpl(TraversableLike.scala:246) at scala.collection.TraversableLike$class.filter(TraversableLike.scala:259) at scala.collection.mutable.ArrayOps$ofRef.filter(ArrayOps.scala:186) at org.apache.spark.deploy.yarn.Client$anonfun$prepareLocalResources$6.apply(Client.scala:531) at org.apache.spark.deploy.yarn.Client$anonfun$prepareLocalResources$6.apply(Client.scala:527) at scala.collection.mutable.ArraySeq.foreach(ArraySeq.scala:74) at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:527) at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:868) at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:170) at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56) at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:156) at org.apache.spark.SparkContext.<init>(SparkContext.scala:509) at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58)

it_duanxiong · ‎08-16-2018

@Sandeep Nemuri,Thank you very much!when I first install oozie,I use mysql.I think it caused by mysql.So i try Derby,But it stay slow!

it_duanxiong · ‎08-16-2018

When I use oozie，I hava install oozie success，But this web loading is very slow，I don‘t know why .Oozie's other function can work,and this web can work too,but web is very slow .Anyone know why this happening?Thank you very much.(or tell me where has Installation Steps and I will go to read(no ambari,just oozie and hadoop))

it_duanxiong · ‎06-07-2017

Can you describe this question more clear?I can't understand this.

it_duanxiong · ‎06-07-2017

Hi,when I register a Hive udf in FunctionRegistry.java,I fund many UDF has been registed in this file,So how i can get the registry function's name list?Like this.So in the fact have the command to realize this? thank you very much.

it_duanxiong · ‎06-07-2017

thank you !I will try this.

it_duanxiong · ‎06-07-2017

Yeah，maybe because the network problem.so i summit the same question twice,sorry!

it_duanxiong · ‎06-06-2017

Ok,thank you very much.I thought that in the fact there is a way to remove it,just I don't know。

it_duanxiong · ‎06-06-2017

Thank you very much,I just want to change the replications of files that have upload to HDFS.So yesterdy,I found that one command can realize that.So thank you very much!

it_duanxiong · ‎06-04-2017

I just want to remove the record!thank you!

it_duanxiong · ‎06-04-2017

when I study HDFS,I encounter one question,I want to make hadoop read the confighration file again,because I have changed the file, when stay hadoop open。and I don't want to reboot the hadoop service! thank you very much!

it_duanxiong · ‎06-04-2017

when I study HDFS,I encounter one question,I want to make hadoop read the confighration file again,because I have changed the file, when stay hadoop open。and I don't want to reboot the hadoop service! thank you very much!

it_duanxiong · ‎11-17-2016

Thank you

it_duanxiong · ‎11-17-2016

when I install ranger-kafka-plugin,I encounter some question. this username is ANONYMOUS,Anybody can help me resolve this problem.

it_duanxiong · ‎10-15-2016

[2016-10-15 17:46:24,378] ERROR Ranger Plugin returned null. Returning false (org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer) [2016-10-15 17:45:59,203] ERROR PolicyRefresher(serviceName=null): failed to refresh policies. Will continue to use last known version of policies (-1) (org.apache.ranger.plugin.util.PolicyRefresher) com.sun.jersey.api.client.ClientHandlerException: java.lang.IllegalArgumentException: URI is not absolute at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151) at com.sun.jersey.api.client.Client.handle(Client.java:648) at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680) at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507) at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:115) at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:217) at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:185) at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:158) Caused by: java.lang.IllegalArgumentException: URI is not absolute at java.net.URI.toURL(URI.java:1095) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149) ... 8 more Have you seen this error?

it_duanxiong · ‎10-10-2016

can you tell me your zookeeper's version?

Online	Offline
Last Visited	‎10-12-2018 02:45 AM

Date Registered	‎10-03-2016 07:23 AM
Last Visited	‎10-12-2018 02:45 AM
Total Messages Posted	51
Total Kudos Received	1

Re: Hi,friend.I want to remote debug oozie workflo...

Re: oozie and spark2 ERROR

Re: spark2 job on oozie

Re: Limit Oozie workflow or node execution time

Re: oozie workflow is stuck at running without err...

Re: oozie workflow is stuck at running without err...

Re: Hi,friend.I want to remote debug oozie workflo...

Hi,friend.I want to remote debug oozie workflow,bu...

Re: oozie4.3 with kerberos: I use oozie in kerbero...

oozie4.3 with kerberos: I use oozie in kerberos h...

Re: oozie and spark2 ERROR

Re: Oozie can't display the graph cause number of ...

Re: Oozie can't display the graph cause number of ...

oozie and spark2 ERROR

Re: One question about Oozie:Web show

Re: spark2 job on oozie

Re: spark2 job on oozie

spark2 job on oozie

Re: One question about Oozie:Web show

One question about Oozie:Web show

Re: Webhdfs REST API - Search for a filename or di...

Hi,when I register a Hive udf in FunctionRegistry....

Re: Hi,How to make hadoop to read the configuratio...

Re: Hi,How to make hadoop to read the configuratio...

Re: Hi,I want to remove hdfs dead datanode in WebU...

Re: Hi,How to make hadoop to read the configuratio...

Hi,I want to remove hdfs dead datanode in WebUi,bu...

Hi,How to make hadoop to read the configuration fi...

Hi,How to make hadoop to read the configuration fi...

Re: ranger-kafka-plugin no kerbers question

ranger-kafka-plugin no kerbers question

Re: ranger-kafka-plugin

Re: ranger-kafka-plugin