Created on 03-06-2017 03:51 AM - edited 09-16-2022 04:11 AM
Hi All,
I am trying to run a Shell action on Oozie to load data to HDFS.
The data gets loaded properly. However, the job fails with the follwoing stderr:
log4j:ERROR Could not find value for key log4j.appender.CLA log4j:ERROR Could not instantiate appender named "CLA". put: `/user/admin/RnR_Demo/Datasets/shellUpload.TestDir/50MB.zip': File exists log4j:ERROR Could not find value for key log4j.appender.CLA log4j:ERROR Could not instantiate appender named "CLA". put: `/user/admin/RnR_Demo/Datasets/shellUpload.TestDir/5MB.zip': File exists log4j:ERROR Could not find value for key log4j.appender.CLA log4j:ERROR Could not instantiate appender named "CLA". put: `/user/admin/RnR_Demo/Datasets/shellUpload.TestDir/CentOS-6.5-x86_64-bin-DVD1(2).zip': File exists rm: cannot remove `oozie-hadoop-conf-1488800663720': Is a directory rm: cannot remove `tmp': Is a directory Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]
here you can see that the hdfs -put command runs successfully since there are files with the same name in the location from a previous run.
however, the error:
log4j:ERROR Could not find value for key log4j.appender.CLA log4j:ERROR Could not instantiate appender named "CLA".
is bugging me since the job fails in spite of data being loaded to HDFS
Please help.
Created 03-09-2017 09:23 AM
Hi anirbandd,
It's likely that you ran into OOZIE-2739. There is a workaround that requires to override the value for oozie.action.shell.setup.hadoop.conf.dir.log4j.content in oozie-site.xml.
If you are using Cloudera Manager, you can easiliy do it via using a Safety valve for Oozie.
Go to Cloudera Manager -> Oozie -> Configuration page and search for oozie-site.xml.
Add the following to "Oozie Server Advanced Configuration Snippet (Safety Valve) for oozie-site.xml" using the XML editor that is better for a newline value carrying config such as the following.
<property>
<name>oozie.action.shell.setup.hadoop.conf.dir.log4j.content</name>
<value>
log4j.rootLogger=INFO,console log4j.appender.console=org.apache.log4j.ConsoleAppender log4j.appender.console.target=System.err log4j.appender.console.layout=org.apache.log4j.PatternLayout
</value>
</property>
Then click Save changes and Re-deploy client configuration (restart Oozie).
Created 03-09-2017 09:23 AM
Hi anirbandd,
It's likely that you ran into OOZIE-2739. There is a workaround that requires to override the value for oozie.action.shell.setup.hadoop.conf.dir.log4j.content in oozie-site.xml.
If you are using Cloudera Manager, you can easiliy do it via using a Safety valve for Oozie.
Go to Cloudera Manager -> Oozie -> Configuration page and search for oozie-site.xml.
Add the following to "Oozie Server Advanced Configuration Snippet (Safety Valve) for oozie-site.xml" using the XML editor that is better for a newline value carrying config such as the following.
<property>
<name>oozie.action.shell.setup.hadoop.conf.dir.log4j.content</name>
<value>
log4j.rootLogger=INFO,console log4j.appender.console=org.apache.log4j.ConsoleAppender log4j.appender.console.target=System.err log4j.appender.console.layout=org.apache.log4j.PatternLayout
</value>
</property>
Then click Save changes and Re-deploy client configuration (restart Oozie).
Created 03-10-2017 03:05 AM