Reply
New Contributor
Posts: 4
Registered: ‎07-24-2016

Re: JA017: Unknown hadoop job

Please help me to resolve this issue. JHS is running in 19888 port but job id of oozie is not showing up. Always getting the below exception whatever tried for fixing this. I really thankful for your help.

 

Message [JA017: Unknown hadoop job [job_local1936976615_0001] associated with action [0000000-160725103332885-oozie-user-W@mapreduce-job]. Failing this action!]
org.apache.oozie.action.ActionExecutorException: JA017: Unknown hadoop job [job_local1936976615_0001] associated with action [0000000-160725103332885-oozie-user-W@mapreduce-job]. Failing this action!
at org.apache.oozie.action.hadoop.JavaActionExecutor.check(JavaActionExecutor.java:1199)
at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1136)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
at org.apache.oozie.command.XCommand.call(XCommand.java:281)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

Posts: 1,507
Kudos: 256
Solutions: 230
Registered: ‎07-31-2013

Re: JA017: Unknown hadoop job

Are you using CM to run this cluster? If yes, is a MR1/MR2 service selected correctly under the Oozie -> Configuration page?

Or, if you aren't using CM, can you ensure you have the Oozie's MR cluster configuration setup right, via the config key of "oozie.service.HadoopAccessorService.hadoop.configurations" in your /etc/oozie/conf/oozie-site.xml?
Backline Customer Operations Engineer
New Contributor
Posts: 4
Registered: ‎07-24-2016

Re: JA017: Unknown hadoop job

Thanks for the quick reply.i had already configured the specified property to /home/hadoop/hadoop-2.6.0/etc/hadoop/ ..
Posts: 1,507
Kudos: 256
Solutions: 230
Registered: ‎07-31-2013

Re: JA017: Unknown hadoop job

Somehow your Oozie instance isn't reading that config right. Its falling back to local execution which indicates the values of your mapred-site.xml and yarn-site.xml under that path are not being read, or are not right in pointing at a running cluster.

Are you able to run regular MR jobs fine using that config directory?

Also, could you pastebin your oozie-site.xml entirely here?
Backline Customer Operations Engineer
New Contributor
Posts: 4
Registered: ‎07-24-2016

Re: JA017: Unknown hadoop job

I couldnt able to run any job from oozie. But i can run fine regular MR jobs via hadoop.

Oozie-site.xml
<configuration>

<!--
Refer to the oozie-default.xml file for the complete list of
Oozie configuration properties and their default values.
-->

<property>
<name>oozie.service.ActionService.executor.ext.classes</name>
<value>
org.apache.oozie.action.email.EmailActionExecutor,
org.apache.oozie.action.hadoop.HiveActionExecutor,
org.apache.oozie.action.hadoop.ShellActionExecutor,
org.apache.oozie.action.hadoop.SqoopActionExecutor,
org.apache.oozie.action.hadoop.DistcpActionExecutor
</value>
</property>

<property>
<name>oozie.service.SchemaService.wf.ext.schemas</name>
<value>
shell-action-0.1.xsd,shell-action-0.2.xsd,shell-action-0.3.xsd,email-action-0.1.xsd,email-action-0.2.xsd,
hive-action-0.2.xsd,hive-action-0.3.xsd,hive-action-0.4.xsd,hive-action-0.5.xsd,sqoop-action-0.2.xsd,
sqoop-action-0.3.xsd,sqoop-action-0.4.xsd,ssh-action-0.1.xsd,ssh-action-0.2.xsd,distcp-action-0.1.xsd,
distcp-action-0.2.xsd,oozie-sla-0.1.xsd,oozie-sla-0.2.xsd
</value>
</property>

<property>
<name>oozie.system.id</name>
<value>oozie-${user.name}</value>
<description>
The Oozie system ID.
</description>
</property>

<property>
<name>oozie.systemmode</name>
<value>NORMAL</value>
<description>
System mode for Oozie at startup.
</description>
</property>

<property>
<name>oozie.service.AuthorizationService.security.enabled</name>
<value>false</value>
<description>
Specifies whether security (user name/admin role) is enabled or not.
If disabled any user can manage Oozie system and manage any job.
</description>
</property>

<property>
<name>oozie.service.PurgeService.older.than</name>
<value>30</value>
<description>
Jobs older than this value, in days, will be purged by the PurgeService.
</description>
</property>

<property>
<name>oozie.service.PurgeService.purge.interval</name>
<value>3600</value>
<description>
Interval at which the purge service will run, in seconds.
</description>
</property>

<property>
<name>oozie.service.CallableQueueService.queue.size</name>
<value>10000</value>
<description>Max callable queue size</description>
</property>

<property>
<name>oozie.service.CallableQueueService.threads</name>
<value>10</value>
<description>Number of threads used for executing callables</description>
</property>

<property>
<name>oozie.service.CallableQueueService.callable.concurrency</name>
<value>3</value>
<description>
Maximum concurrency for a given callable type.
Each command is a callable type (submit, start, run, signal, job, jobs, suspend,resume, etc).
Each action type is a callable type (Map-Reduce, Pig, SSH, FS, sub-workflow, etc).
All commands that use action executors (action-start, action-end, action-kill and action-check) use
the action type as the callable type.
</description>
</property>

<property>
<name>oozie.service.coord.normal.default.timeout
</name>
<value>120</value>
<description>Default timeout for a coordinator action input check (in minutes) for normal job.
-1 means infinite timeout</description>
</property>

<property>
<name>oozie.db.schema.name</name>
<value>oozie</value>
<description>
Oozie DataBase Name
</description>
</property>

<property>
<name>oozie.service.JPAService.create.db.schema</name>
<value>false</value>
<description>
Creates Oozie DB.

If set to true, it creates the DB schema if it does not exist. If the DB schema exists is a NOP.
If set to false, it does not create the DB schema. If the DB schema does not exist it fails start up.
</description>
</property>

<property>
<name>oozie.service.JPAService.jdbc.driver</name>
<value>org.apache.derby.jdbc.EmbeddedDriver</value>
<description>
JDBC driver class.
</description>
</property>

<property>
<name>oozie.service.JPAService.jdbc.url</name>
<value>jdbc:derby:${oozie.data.dir}/${oozie.db.schema.name}-db;create=true</value>
<description>
JDBC URL.
</description>
</property>

<property>
<name>oozie.service.JPAService.jdbc.username</name>
<value>sa</value>
<description>
DB user name.
</description>
</property>

<property>
<name>oozie.service.JPAService.jdbc.password</name>
<value> </value>
<description>
DB user password.

IMPORTANT: if password is emtpy leave a 1 space string, the service trims the value,
if empty Configuration assumes it is NULL.
</description>
</property>

<property>
<name>oozie.service.JPAService.pool.max.active.conn</name>
<value>10</value>
<description>
Max number of connections.
</description>
</property>

<property>
<name>oozie.service.HadoopAccessorService.kerberos.enabled</name>
<value>false</value>
<description>
Indicates if Oozie is configured to use Kerberos.
</description>
</property>

<property>
<name>local.realm</name>
<value>LOCALHOST</value>
<description>
Kerberos Realm used by Oozie and Hadoop. Using 'local.realm' to be aligned with Hadoop configuration
</description>
</property>

<property>
<name>oozie.service.HadoopAccessorService.keytab.file</name>
<value>${user.home}/oozie.keytab</value>
<description>
Location of the Oozie user keytab file.
</description>
</property>

<property>
<name>oozie.service.HadoopAccessorService.kerberos.principal</name>
<value>${user.name}/localhost@${local.realm}</value>
<description>
Kerberos principal for Oozie service.
</description>
</property>

<property>
<name>oozie.service.HadoopAccessorService.jobTracker.whitelist</name>
<value> </value>
<description>
Whitelisted job tracker for Oozie service.
</description>
</property>

<property>
<name>oozie.service.HadoopAccessorService.nameNode.whitelist</name>
<value> </value>
<description>
Whitelisted job tracker for Oozie service.
</description>
</property>

<property>
<name>oozie.service.HadoopAccessorService.hadoop.configurations</name>
<value>*=/home/hadoop/hadoop-2.6.0/etc/hadoop/</value>
<description>
Comma separated AUTHORITY=HADOOP_CONF_DIR, where AUTHORITY is the HOST:PORT of
the Hadoop service (JobTracker, HDFS). The wildcard '*' configuration is
used when there is no exact match for an authority. The HADOOP_CONF_DIR contains
the relevant Hadoop *-site.xml files. If the path is relative is looked within
the Oozie configuration directory; though the path can be absolute (i.e. to point
to Hadoop client conf/ directories in the local filesystem.
</description>
</property>

<property>
<name>oozie.service.WorkflowAppService.system.libpath</name>
<value>hdfs:///home/oozie/oozie-bin/share/lib</value>
<description>
System library path to use for workflow applications.
This path is added to workflow application if their job properties sets
the property 'oozie.use.system.libpath' to true.
</description>
</property>

<property>
<name>use.system.libpath.for.mapreduce.and.pig.jobs</name>
<value>false</value>
<description>
If set to true, submissions of MapReduce and Pig jobs will include
automatically the system library path, thus not requiring users to
specify where the Pig JAR files are. Instead, the ones from the system
library path are used.
</description>
</property>

<property>
<name>oozie.authentication.type</name>
<value>simple</value>
<description>
Defines authentication used for Oozie HTTP endpoint.
Supported values are: simple | kerberos | #AUTHENTICATION_HANDLER_CLASSNAME#
</description>
</property>

<property>
<name>oozie.authentication.token.validity</name>
<value>36000</value>
<description>
Indicates how long (in seconds) an authentication token is valid before it has
to be renewed.
</description>
</property>

<property>
<name>oozie.authentication.signature.secret</name>
<value>oozie</value>
<description>
The signature secret for signing the authentication tokens.
If not set a random secret is generated at startup time.
In order to authentiation to work correctly across multiple hosts
the secret must be the same across al the hosts.
</description>
</property>

<property>
<name>oozie.authentication.cookie.domain</name>
<value></value>
<description>
The domain to use for the HTTP cookie that stores the authentication token.
In order to authentiation to work correctly across multiple hosts
the domain must be correctly set.
</description>
</property>

<property>
<name>oozie.authentication.simple.anonymous.allowed</name>
<value>true</value>
<description>
Indicates if anonymous requests are allowed.
This setting is meaningful only when using 'simple' authentication.
</description>
</property>

<property>
<name>oozie.authentication.kerberos.principal</name>
<value>HTTP/localhost@${local.realm}</value>
<description>
Indicates the Kerberos principal to be used for HTTP endpoint.
The principal MUST start with 'HTTP/' as per Kerberos HTTP SPNEGO specification.
</description>
</property>

<property>
<name>oozie.authentication.kerberos.keytab</name>
<value>${oozie.service.HadoopAccessorService.keytab.file}</value>
<description>
Location of the keytab file with the credentials for the principal.
Referring to the same keytab file Oozie uses for its Kerberos credentials for Hadoop.
</description>
</property>

<property>
<name>oozie.authentication.kerberos.name.rules</name>
<value>DEFAULT</value>
<description>
The kerberos names rules is to resolve kerberos principal names, refer to Hadoop's
KerberosName for more details.
</description>
</property>

<!-- Proxyuser Configuration -->

<!--

<property>
<name>oozie.service.ProxyUserService.proxyuser.#USER#.hosts</name>
<value>*</value>
<description>
List of hosts the '#USER#' user is allowed to perform 'doAs'
operations.

The '#USER#' must be replaced with the username o the user who is
allowed to perform 'doAs' operations.

The value can be the '*' wildcard or a list of hostnames.

For multiple users copy this property and replace the user name
in the property name.
</description>
</property>

<property>
<name>oozie.service.ProxyUserService.proxyuser.#USER#.groups</name>
<value>*</value>
<description>
List of groups the '#USER#' user is allowed to impersonate users
from to perform 'doAs' operations.

The '#USER#' must be replaced with the username o the user who is
allowed to perform 'doAs' operations.

The value can be the '*' wildcard or a list of groups.

For multiple users copy this property and replace the user name
in the property name.
</description>
</property>

-->

</configuration>
New Contributor
Posts: 4
Registered: ‎07-24-2016

Re: JA017: Unknown hadoop job

even yarn and mapreduce logs are not shown. There is no job id in JHS for map-reduce jobs and oozie map-reduce as well. 

 

In yarn-site.xml ***********

<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>

<property>
<name>yarn.nodemanager.remote-app-log-dir</name>
<value>/var/log/hadoop-yarn/apps</value>
</property>

 

 

In mapred-site.xml ******

 

<name>mapreduce.jobhistory.done-dir</name>
<value>/mr-history/done </value>
</property>
<property>
<name>mapreduce.jobhistory.intermediate-done-dir</name>
<value>/mr-history/tmp</value>

<name>yarn.app.mapreduce.am.staging-dir</name>
<value>/user</value>
</property>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>

 

But none of the logs has been created in hdfs. This seems to be any permission issue? Please help me to make oozie work. Really thankful for your help!!!!

 

 

Highlighted
New Contributor
Posts: 1
Registered: ‎06-22-2017

Re: JA017: Unknown hadoop job

Hi Guys, I was also facing same issue while submitting jobs to Oozie on my local system. I tried to find any solutions on google, but havent succeeded.

 

Then i got to know that oozie always execute a workflow through mapreduce and i tried to run a MAPRED 'hello world' and it got failed.

 

Now configured few properties for MAPRED to works in mapred-site.xml and yarn-site.xml as:

 

For yarn-site.xml:

 

<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
</property>

<property>
    <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
  
<property>
	<name>yarn.log-aggregation-enable</name>
	<value>true</value>
</property>

For mapred-site.xml:

 

<property>
	<name>mapreduce.jobtracker.address</name>
	<value>HOST:PORT</value>
</property>

<property>
	<name>mapreduce.jobtracker.http.address</name>
	<value>HOST:PORT</value>
</property>

<property>
	<name>mapreduce.tasktracker.report.address</name>
	<value>127.0.0.1:0</value>
</property>

<property>
	<name>mapreduce.tasktracker.http.address</name>
	<value>0.0.0.0:50060</value>
</property>

<property>
	<name>mapreduce.job.queuename</name>
	<value>default</value>
</property>

<property>
	<name> mapreduce.framework.name</name>
	<value>yarn</value>
</property>

<property> 
	<name>mapreduce.jobhistory.address</name>
	<value>localhost:10020</value> 
</property>

<property> 
	<name>mapreduce.jobhistory.webapp.address</name>
	<value>localhost:19888</value> 
</property>

And make sure job history server is up and running and values of the properties should be unique on all the nodes of the cluster: 

 

mapreduce.jobhistory.done-dir 
mapreduce.jobhistory.intermediate-done-dir

After these changes, i am able to submit oozie workflow on my local.

Announcements