Sqoop job running in oozie getting killed with err message Container exited with a non-zero exit code 137

[#1]

sqoop jobs running in oozie get killed randomly with error

There is no one really killing the job so why do I see :

Killed by external signal<br>

I get this error in task diagnostic log

Container exited with a non-zero exit code 137
Killed by external signal

Under diagnostics, I keep getting:

Application killed by user.

Heap Memory allocated to Sqoop is 2GB.

we started getting this error out of nowhere and now it is seldom that my oozie workflow with all sqoop jobs in it completes without this error. On every 2nd or 4th job, I get this error.

I don't see any other errors anywhere. Can I please get some pointers on this? How do I fix this?

I also see some of them failing with:

2018-04-16 22:25:21,494 [main] ERROR org.apache.sqoop.tool.ImportTool  - Import failed: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory <a href="link">hdfs://ip:8020/user/hue/folder/hots/2017-04-16--08--30--00/table_hsn_mapping</a> already exists

All my jobs in oozie are ecxecuting with:

job --meta-connect jdbc:hsqldb:hsql://ip:16000/sqoop --exec table_hsn_mapping -- --warehouse-dir folder/Snapshots/${DATE}

So, I never used to get FileAlreadyExists error because of using warehouse-dir folder. Somehow I started getting it 2 days back and none of it makes sense to me right now. Any help would be appreciated.

[#2]

@Simran Kaur

Can you please share your workflow/job.properties and complete Oozie launcher logs?

[#3]

I see this message in stdout tab in oozie:

Found [1] Map-Reduce jobs from this launcher
Killing existing jobs and starting over:
2018-04-18 14:45:13,569 [main] INFO  org.apache.hadoop.yarn.client.RMProxy  - Connecting to ResourceManager at ip-172-31-4-192.ap-south-1.compute.internal/172.31.4.192:8032
Killing job [application_1523964434795_4483] ... 2018-04-18 14:45:13,776 [main] INFO  org.apache.hadoop.yarn.client.api.impl.YarnClientImpl  - Killed application application_1523964434795_4483 

In stderr:

          
            Log Upload Time: Wed Apr 18 14:45:23 +0530 2018
          
            Log Length: 288
          Note: /tmp/sqoop-yarn/compile/d443da40c930f2217e70e38a82730dc0/fabric_po.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Intercepting System.exit(1) 

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]

[#4]

Hi Kuldeep,

Good day...,Details are as attached.workflow.xmlwfl-logs.txtresourcemanager-logs.txt

Job.Properties

nameNode=hdfs://ssehdp101.biz:8020

jobTracker=ssehdp102.biz:8050

queueName=Process

examplesRoot=ravi

oozie.use.system.libpath=true

oozie.libpath=${nameNode}/user/oozie/share/lib/lib_20170922104734/

oozie.wf.rerun.failnodes=true

oozie.wf.application.path=${nameNode}/user/${user.name}/ravi/workflow.xml

oozie.action.sharelib.for.sqoop=sqoop,hive