Member since
10-03-2016
51
Posts
1
Kudos Received
3
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
779 | 09-14-2018 01:30 AM | |
943 | 08-24-2018 03:37 AM | |
1596 | 08-18-2018 02:19 AM |
10-10-2018
12:44 AM
I want this function too.Current version do not have this function to finish this.
... View more
09-17-2018
01:43 AM
oozie can successed schedule the other mission , i.e. mapreduce?
... View more
09-16-2018
06:42 AM
I agree with you!you are right about this problem,First,have to try use spark-sumit to run this App,Then can use oozie go to schedule.
... View more
09-14-2018
01:30 AM
1 Kudo
Just need to add this tag in wf.xml <property>
<name>oozie.launcher.mapred.child.java.opts</name>
<value>-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=8000</value>
</property>
... View more
09-13-2018
12:27 PM
Hi,friend.I want to remote debug oozie workflow,but I don't know how to do it.I need some help.Thank you very much. I use oozie.sh jpda start,and use my IDEA connected,but i can't submit wf and the oozie process is on ,so somebody can tell me how to do next?
... View more
Labels:
- Labels:
-
Apache Oozie
09-06-2018
01:31 PM
run:oozie admin -oozie http://localhost:11000/oozie -status have the question
... View more
09-06-2018
01:28 PM
wrong log: Error: IO_ERROR : java.io.IOException: Error while connecting Oozie server. No of retries = 1. Exception = Could not authenticate, Authentication failed, status: 302, message: Found oozie-site.xml: <property>
<name>oozie.service.HadoopAccessorService.kerberos.enabled</name>
<value>true</value>
</property>
<property>
<name>local.realm</name>
<value>HADOOP.COM</value>
</property>
<property>
<name>oozie.service.HadoopAccessorService.keytab.file</name>
<value>/usr/lib/cluster001/oozie-4.3.0/conf/oozie-http.keytab</value>
</property>
<property>
<name>oozie.service.HadoopAccessorService.kerberos.principal</name>
<value>oozie/ds1@HADOOP.COM</value>
</property>
<property>
<name>oozie.authentication.type</name>
<value>kerberos</value>
</property>
<property>
<name>oozie.authentication.kerberos.principal</name>
<value>HTTP/ds1@HADOOP.COM</value>
</property>
<property>
<name>oozie.authentication.kerberos.name.rules</name>
<value>DEFAULT</value>
</property>
keytab:(ds1上运行oozie-server,I have test this tap,It is successd) [root@ds1 conf]# klist -e -k -t oozie-http.keytab
Keytab name: FILE:oozie-http.keytab
KVNO Timestamp Principal
---- ----------------- --------------------------------------------------------
1 09/05/18 14:56:16 oozie/ds1@HADOOP.COM (aes256-cts-hmac-sha1-96)
1 09/05/18 14:56:16 oozie/ds1@HADOOP.COM (aes128-cts-hmac-sha1-96)
2 09/05/18 14:56:16 HTTP/ds1@HADOOP.COM (aes256-cts-hmac-sha1-96)
2 09/05/18 14:56:16 HTTP/ds1@HADOOP.COM (aes128-cts-hmac-sha1-96)
1 09/06/18 19:43:58 oozie/ds1@HADOOP.COM (aes256-cts-hmac-sha1-96)
1 09/06/18 19:43:58 oozie/ds1@HADOOP.COM (aes128-cts-hmac-sha1-96)
2 09/06/18 19:43:58 HTTP/ds1@HADOOP.COM (aes256-cts-hmac-sha1-96)
2 09/06/18 19:43:58 HTTP/ds1@HADOOP.COM (aes128-cts-hmac-sha1-96)
... View more
Labels:
- Labels:
-
Apache Oozie
08-24-2018
03:37 AM
This problem have to add spark.yarn.jars in sparkopts.and It have to start with"hdfs://"
... View more
08-24-2018
03:35 AM
It's my linux problem.
... View more
08-23-2018
03:46 AM
Hi,I have the same question,same error with one 1 action,I try your method,but it didn't work!
... View more
08-20-2018
11:40 AM
version: oozie:4.3 spark2.1 2018-08-20 19:33:23,998 [uber-SubtaskRunner] ERROR org.apache.spark.SparkContext - Error initializing SparkContext.
java.lang.NullPointerException
at scala.collection.mutable.ArrayOps$ofRef$.newBuilder$extension(ArrayOps.scala:190)
at scala.collection.mutable.ArrayOps$ofRef.newBuilder(ArrayOps.scala:186)
at scala.collection.TraversableLike$class.filterImpl(TraversableLike.scala:246)
at scala.collection.TraversableLike$class.filter(TraversableLike.scala:259)
at scala.collection.mutable.ArrayOps$ofRef.filter(ArrayOps.scala:186)
at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:531)
at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:527)
at scala.collection.mutable.ArraySeq.foreach(ArraySeq.scala:74)
at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:527)
at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:868)
at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:170)
... View more
Labels:
- Labels:
-
Apache Oozie
08-18-2018
02:23 AM
Yaeh,I don't konw why,I change another cluster,every thing is fine.so thank you very much!
... View more
08-18-2018
02:19 AM
When use spark2.1,We have to add oozie-sharelib-spark-4.2.0.jar in spark2
... View more
08-17-2018
12:31 PM
when I use oozie to schedule spark2 on yarn.I have e pproblem : Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, null
java.lang.NullPointerException
at scala.collection.mutable.ArrayOps$ofRef$.newBuilder$extension(ArrayOps.scala:190)
at scala.collection.mutable.ArrayOps$ofRef.newBuilder(ArrayOps.scala:186)
at scala.collection.TraversableLike$class.filterImpl(TraversableLike.scala:246)
at scala.collection.TraversableLike$class.filter(TraversableLike.scala:259)
at scala.collection.mutable.ArrayOps$ofRef.filter(ArrayOps.scala:186)
at org.apache.spark.deploy.yarn.Client$anonfun$prepareLocalResources$6.apply(Client.scala:531)
at org.apache.spark.deploy.yarn.Client$anonfun$prepareLocalResources$6.apply(Client.scala:527)
at scala.collection.mutable.ArraySeq.foreach(ArraySeq.scala:74)
at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:527)
at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:868)
at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:170)
at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56)
at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:156)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:509)
at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58)
... View more
Labels:
- Labels:
-
Apache Oozie
-
Apache Spark
08-16-2018
03:29 PM
@Sandeep Nemuri,Thank you very much!when I first install oozie,I use mysql.I think it caused by mysql.So i try Derby,But it stay slow!
... View more
08-16-2018
11:18 AM
When I use oozie,I hava install oozie success,But this web loading is very slow,I don‘t know why .Oozie's other function can work,and this web can work too,but web is very slow .Anyone know why this happening?Thank you very much.(or tell me where has Installation Steps and I will go to read(no ambari,just oozie and hadoop))
... View more
Labels:
- Labels:
-
Apache Oozie
06-07-2017
08:48 AM
Can you describe this question more clear?I can't understand this.
... View more
06-07-2017
08:32 AM
Hi,when I register a Hive udf in FunctionRegistry.java,I fund many UDF has been registed in this file,So how i can get the registry function's name list?Like this.So in the fact have the command to realize this? thank you very much.
... View more
Labels:
- Labels:
-
Apache Hive
06-07-2017
12:44 AM
thank you !I will try this.
... View more
06-07-2017
12:44 AM
Yeah,maybe because the network problem.so i summit the same question twice,sorry!
... View more
06-06-2017
01:13 PM
Ok,thank you very much.I thought that in the fact there is a way to remove it,just I don't know。
... View more
06-06-2017
12:53 PM
Thank you very much,I just want to change the replications of files that have upload to HDFS.So yesterdy,I found that one command can realize that.So thank you very much!
... View more
06-04-2017
03:18 PM
I just want to remove the record!thank you!
... View more
- Tags:
- Hadoop Core
- HDFS
Labels:
- Labels:
-
Apache Hadoop
06-04-2017
09:28 AM
when I study HDFS,I encounter one question,I want to make hadoop read the confighration file again,because I have changed the file, when stay hadoop open。and I don't want to reboot the hadoop service! thank you very much!
... View more
Labels:
- Labels:
-
Apache Hadoop
06-04-2017
09:28 AM
when I study HDFS,I encounter one question,I want to make hadoop read the confighration file again,because I have changed the file, when stay hadoop open。and I don't want to reboot the hadoop service! thank you very much!
... View more
Labels:
- Labels:
-
Apache Hadoop
11-17-2016
03:31 PM
Thank you
... View more
11-17-2016
08:44 AM
when I install ranger-kafka-plugin,I encounter some question. this username is ANONYMOUS,Anybody can help me resolve this problem.
... View more
Labels:
- Labels:
-
Apache Kafka
-
Apache Ranger
10-15-2016
10:06 AM
[2016-10-15 17:46:24,378] ERROR Ranger Plugin returned null. Returning
false
(org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer)
[2016-10-15 17:45:59,203] ERROR PolicyRefresher(serviceName=null): failed to refresh policies. Will continue to use last known version of policies (-1) (org.apache.ranger.plugin.util.PolicyRefresher)
com.sun.jersey.api.client.ClientHandlerException: java.lang.IllegalArgumentException: URI is not absolute
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:115)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:217)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:185)
at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:158)
Caused by: java.lang.IllegalArgumentException: URI is not absolute
at java.net.URI.toURL(URI.java:1095)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
... 8 more
Have you seen this error?
... View more