About it_duanxiong

channa · ‎12-20-2019

https://github.com/apache/oozie/blob/9c288fe5cea6f2fbbae76f720b9e215acdd07709/webapp/src/main/webapp/oozie-console.js#L384

it_duanxiong · ‎10-08-2018

it_duanxiong · ‎09-25-2018

how to remote debug oozie on spark2?

it_duanxiong · ‎09-14-2018

Just need to add this tag in wf.xml <property> <name>oozie.launcher.mapred.child.java.opts</name> <value>-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=8000</value> </property>

vmurakami · ‎09-17-2018

@A C Just to understand, did you run the spark submit using yarn cluster as master/deploy mode? If so, let's try to check the job properties for the following parameter: ${resourceManager} Also, here it is another example regarding pyspark + oozie (using shell to submit spark). https://github.com/hgrif/oozie-pyspark-workflow Hope this helps

it_duanxiong · ‎09-06-2018

run：oozie admin -oozie http://localhost:11000/oozie -status have the question

it_duanxiong · ‎08-24-2018

This problem have to add spark.yarn.jars in sparkopts.and It have to start with"hdfs://"

it_duanxiong · ‎08-20-2018

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] log4j:ERROR Could not find value for key log4j.appender.CLA log4j:ERROR Could not instantiate appender named "CLA". log4j:ERROR Could not find value for key log4j.appender.CLA log4j:ERROR Could not instantiate appender named "CLA". Using properties file: null When I use oozie on spark2,I run example "SparkFileCopy",It have some problem.Thank you. version: oozie 4.3 spark 2.1

it_duanxiong · ‎08-18-2018

When I use oozie on spark21.It hava one error: Library directory '/data/hadoop/usercache/root/appcache/application_1533889422422_0065/container_1533889422422_0065_01_000001/assembly/target/scala-2.11/jars' does not exist; make sure Spark is built. Thanks!

nikil_katturi · ‎08-21-2018

@Felix Albani i am facing the issues while i am running the job in oozie . Below is the error 2018-08-21 14:27:16,632 ERROR [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster java.lang.IllegalArgumentException: Invalid ContainerId: container_e63_1532612094367_17160_02_000001 at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:182) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1343)

it_duanxiong · ‎08-18-2018

Yaeh,I don't konw why,I change another cluster,every thing is fine.so thank you very much!

it_duanxiong · ‎08-16-2018

When I run oozie's example map-reduce.In yarn application,It shows this mission had successd,But in oozie ,this mission always running？I don't know how to resolve it.Anyonr has some idea about this question,Thank you very much!

it_duanxiong · ‎07-25-2017

When I do this,I found many source files are synthesized into a single file，I don't want like this,I just want to realize one file to another file,one to one.So anyone have function to realize this,Thank you very much!

ssubhas · ‎06-14-2017

@Xiong Duan Try using query 'show functions' for listing all available functions under Hive.

it_duanxiong · ‎06-06-2017

Ok,thank you very much.I thought that in the fact there is a way to remove it,just I don't know。

agillan · ‎06-06-2017

You're very welcome! Yes, for this the setrep command can change the replication factor for existing files and there is no need to change the value globally. For others' reference, the command is: hdfs dfs -setrep [-R] [-w] <numReplicas> <path>

it_duanxiong · ‎06-07-2017

thank you !I will try this.

it_duanxiong · ‎11-17-2016

Thank you

sshimpi · ‎10-31-2016

@Xiong Duan Can you attach the screenshot of the Kafka Repository configuration here ? Also make sure you are using username as Kerberos user principal in Kafka repository config. Also please try to enable ranger debug for for information. Please do check ranger admin and kafka server logs to dig more.

it_duanxiong · ‎10-28-2016

Yes,I install kafka with kerberos success,But in solr--access don't have records.Do you know why?

gborad · ‎10-17-2016

@Xiong Duan, I meant the file system permissions of the plugin files. Also, please see the more comprehensive answer from @Ramesh Mani.

it_duanxiong · ‎10-15-2016

[2016-10-15 17:46:24,378] ERROR Ranger Plugin returned null. Returning false (org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer) [2016-10-15 17:45:59,203] ERROR PolicyRefresher(serviceName=null): failed to refresh policies. Will continue to use last known version of policies (-1) (org.apache.ranger.plugin.util.PolicyRefresher) com.sun.jersey.api.client.ClientHandlerException: java.lang.IllegalArgumentException: URI is not absolute at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151) at com.sun.jersey.api.client.Client.handle(Client.java:648) at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680) at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507) at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:115) at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:217) at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:185) at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:158) Caused by: java.lang.IllegalArgumentException: URI is not absolute at java.net.URI.toURL(URI.java:1095) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149) ... 8 more Have you seen this error?

it_duanxiong · ‎10-08-2016

I don't understand kafka ranger is supported only on secure cluster.Can you explain this for me? thank you.

Online	Offline
Last Visited	‎10-12-2018 02:45 AM

Date Registered	‎10-03-2016 07:23 AM
Last Visited	‎10-12-2018 02:45 AM
Total Messages Posted	51
Total Kudos Received	1

Re: Hi,friend.I want to remote debug oozie workflo...

Re: oozie and spark2 ERROR

Re: spark2 job on oozie

Re: Oozie can't display the graph cause number of ...

Hi,How to use IDEA local debug oozie resourceCode...

how to remote debug oozie on spark2?

Re: Hi,friend.I want to remote debug oozie workflo...

Re: oozie workflow is stuck at running without err...

Re: oozie4.3 with kerberos: I use oozie in kerbero...

Re: oozie and spark2 ERROR

Error in oozie on spark2

oozie on spark2.1 Error: make sure Spark is built.

Re: spark2 job on oozie

Re: One question about Oozie:Web show

Oozie have a wrong info :Error : E0303: Invalid pa...

About flume,I want to use flume to archieve copy f...

Re: Hi,when I register a Hive udf in FunctionRegis...

Re: Hi,I want to remove hdfs dead datanode in WebU...

Re: Hi,How to make hadoop to read the configuratio...

Re: Hi,How to make hadoop to read the configuratio...

Re: ranger-kafka-plugin no kerbers question

Re: ranger-kafka-plugin did't show in ranger-->au...

Re: ranger-kafka-plugin didn't success

Re: ranger-kafka-plugin

Re: ranger-kafka-plugin

Re: kafka-plugin in ranger didn't success