Have deployed the cluster with 3 data nodes. YARN/MapReduce2/HDFS version is 2.7.3 on HDP.
While running teragen and Gobblin the following Yarn errors get reported in the logs. Errors get reported only when the map tasks defined for the job less than or equals to the number of data nodes in the cluster.
For Teragen -Dmapreduce.job.maps=4
For Gobblin mr.job.max.mappers=4
There are no errors if the map tasks(splits) are <= number of data nodes.
Note: There is no functionality issue
Could someone please help me to avoid the errors in the logs?
2018-08-16 06:54:05,681 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container id container_1534394833079_0012_01_000006
2018-08-16 05:00:50,138 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container id container_1534394833079_0001_01_000055
2018-08-16 05:00:50,138 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_1534394833079_0001_01_000054
2018-08-16 05:00:50,138 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container id container_1534394833079_0001_01_000054
2018-08-16 05:00:50,138 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_1534394833079_0001_01_000053
2018-08-16 05:00:50,138 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container id container_1534394833079_0001_01_000053
