Member since
07-19-2017
8
Posts
1
Kudos Received
0
Solutions
09-07-2017
06:20 AM
This Question is no longer available. 🙂
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Hive
08-16-2017
01:52 PM
I have kerborized HDP cluster and separate non-kerborized HDF cluster. I want to communicate with HDP kafka. I made all the changes related to communicate with kerborized kafka in HDF. Below is the error i am facing:
... View more
Labels:
08-15-2017
10:10 AM
Although kafka port is 6667 but when i execute below command it will pick message and give below error after a while: ./kafka-console-producer.sh --broker-list localhost:9092 --topic test ERROR Error when sending message to topic test with key: null, value: 6 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.
... View more
08-14-2017
06:12 AM
NetUtil.py:67 - Connecting to https://Server:8440/ca
NetUtil.py:93 - [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579)
NetUtil.py:94 - SSLError: Failed to connect. Please check openssl library versions. In order to increase compute of AWS DataNode instance we have stopped the data node instance and increase the compute with out stopping the Node services. When we restarted the instance then the heartbeat of the node went down with above Ambari agent error on node.
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hadoop
08-13-2017
04:17 PM
raise Fail("Applying %s failed, parent directory %s doesn't exist" % (self.resource, dirname))
resource_management.core.exceptions.Fail: Applying File['/var/lib/ambari-agent/tmp/changeUid.sh'] failed, parent directory /var/lib/ambari-agent/tmp doesn't exist
Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-11717.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-11717.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1']
Ambari version is 2.5 This issue is appearing on master node. All the services on master node is down. Initially one of the data node was not able to make connection with master node. It seems that firewalld was not disable. we install the service and disabled it. and restarted ambari. After restarting ambari all the services on master node went down with above error. As this is master node so we wouldn't able to re-set ambari or reinstall it.
... View more
Labels:
- Labels:
-
Apache Ambari
07-28-2017
08:18 AM
1 Kudo
I have added new queue developer and given a capacity 40 %. Below is my change: yarn.scheduler.capacity.maximum-am-resource-percent=0.2
yarn.scheduler.capacity.maximum-applications=10000
yarn.scheduler.capacity.node-locality-delay=40
yarn.scheduler.capacity.root.accessible-node-labels=*
yarn.scheduler.capacity.root.acl_administer_queue=*
yarn.scheduler.capacity.root.capacity=100
yarn.scheduler.capacity.root.default.acl_submit_applications=root
yarn.scheduler.capacity.root.default.capacity=60
yarn.scheduler.capacity.root.default.maximum-capacity=100
yarn.scheduler.capacity.root.default.state=RUNNING
yarn.scheduler.capacity.root.default.user-limit-factor=1
yarn.scheduler.capacity.root.queues=default,developers
yarn.scheduler.capacity.queue-mappings-override.enable=false
yarn.scheduler.capacity.root.default.priority=0
yarn.scheduler.capacity.root.developers.acl_administer_queue=*
yarn.scheduler.capacity.root.developers.acl_submit_applications=hdfs
yarn.scheduler.capacity.root.developers.capacity=40
yarn.scheduler.capacity.root.developers.maximum-capacity=40
yarn.scheduler.capacity.root.developers.minimum-user-limit-percent=100
yarn.scheduler.capacity.root.developers.ordering-policy=fifo
yarn.scheduler.capacity.root.developers.priority=0
yarn.scheduler.capacity.root.developers.state=RUNNING
yarn.scheduler.capacity.root.developers.user-limit-factor=1
yarn.scheduler.capacity.root.priority=0 When i execute query from hdfs user that belong to developer group then it is going in default queue instead of developer. Please note that i am using hdp 2.5 and ambari 2.6 version,
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache YARN