Cloudera Community
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
sivasankar_boom
user rank
Explorer

Re: How can I configure Kafka (HDP 2.3) on AWS to ...

by Explorer in Support Questions
‎08-17-2018 04:19 AM
‎08-17-2018 04:19 AM
Can make it work with below config With default port advertised.host.name= 34.207.113.18(public ip of kafka node) advertised.port= 6667(Listener port) host.name= 34.207.113.18(public ip of kafka node) Without default port listener.security.protocol.map=INGEST:PLAINTEXT,REPLICATION:PLAINTEXT,INTERNAL_PLAINTEXT:PLAINTEXT listeners=REPLICATION://10.0.0.207:9093,INTERNAL_PLAINTEXT://10.0.0.207:9092 advertised.listeners=REPLICATION://34.207.113.18:9093,INTERNAL_PLAINTEXT://34.207.113.18:9092 inter.broker.listener.name=REPLICATION advertised.port= 9094 ... View more

Re: ERROR: Container complete event for unknown co...

by Explorer in Support Questions
‎08-16-2018 08:48 PM
‎08-16-2018 08:48 PM
@Predrag Minovic Have deployed the cluster with 3 data nodes. YARN/MapReduce2/HDFS version is 2.7.3 on HDP. While running teragen and Gobblin the following Yarn errors get reported in the logs. Errors get reported only when the map tasks defined for the job less than or equals to the number of data nodes in the cluster. For Teragen -Dmapreduce.job.maps=4 For Gobblin mr.job.max.mappers=4 There are no errors if the map tasks(splits) are <= number of data nodes. 2018-08-16 06:54:05,681 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container id container_1534394833079_0012_01_000006 2018-08-16 05:00:50,138 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container id container_1534394833079_0001_01_000055 2018-08-16 05:00:50,138 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_1534394833079_0001_01_000054 2018-08-16 05:00:50,138 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container id container_1534394833079_0001_01_000054 2018-08-16 05:00:50,138 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_1534394833079_0001_01_000053 2018-08-16 05:00:50,138 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container id container_1534394833079_0001_01_000053 Please let me know how to avoid these error in the YARN logs. ... View more

RMContainerAllocator: Container complete event for...

by Explorer in Support Questions
‎08-16-2018 08:47 PM
‎08-16-2018 08:47 PM
Have deployed the cluster with 3 data nodes. YARN/MapReduce2/HDFS version is 2.7.3 on HDP. While running teragen and Gobblin the following Yarn errors get reported in the logs. Errors get reported only when the map tasks defined for the job less than or equals to the number of data nodes in the cluster. For Teragen -Dmapreduce.job.maps=4 For Gobblin mr.job.max.mappers=4 There are no errors if the map tasks(splits) are <= number of data nodes. Note: There is no functionality issue Could someone please help me to avoid the errors in the logs? 2018-08-16 06:54:05,681 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container id container_1534394833079_0012_01_000006 2018-08-16 05:00:50,138 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container id container_1534394833079_0001_01_000055 2018-08-16 05:00:50,138 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_1534394833079_0001_01_000054 2018-08-16 05:00:50,138 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container id container_1534394833079_0001_01_000054 2018-08-16 05:00:50,138 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_1534394833079_0001_01_000053 2018-08-16 05:00:50,138 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Container complete event for unknown container id container_1534394833079_0001_01_000053 ... View more
Labels:

Re: How can I configure Kafka (HDP 2.3) on AWS to ...

by Explorer in Support Questions
‎06-29-2018 10:29 AM
‎06-29-2018 10:29 AM
Thanks Riccardo, it worked for me. ... View more
Contact Me
Online
Offline
Last Visited
‎08-20-2018 06:06 AM
Community Statistics
Member Since ‎06-29-2018 10:22 AM
Last Visited ‎08-20-2018 06:06 AM
Posts 4