Member since
10-28-2016
392
Posts
7
Kudos Received
20
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2795 | 03-12-2018 02:28 AM | |
4345 | 12-18-2017 11:41 PM | |
3102 | 07-17-2017 07:01 PM | |
2158 | 07-13-2017 07:20 PM | |
6513 | 07-12-2017 08:31 PM |
11-28-2016
08:03 PM
@Ayub Khan, thanks, i was able to add the user kafka1, kafka2 - to KDC and get Kerberos token for them. However, now i'm facing issue wherein i'm not able to publish to Kafka topic, even when i'm logged in as root (where i believe it uses service user - kafka) When i publish to the topic, i'm getting this error -> ------------------------------------------------------------- [root@sandbox libs]# $KAFKA_HOME/bin/kafka-console-producer.sh --broker-list sandbox.hortonworks.com:6667
--topic kafka1_topic2 --security-protocol SASL_PLAINTEXT hello
[2016-11-28 06:44:03,499] WARN Error while fetching metadata
[{TopicMetadata for topic kafka1_topic2 -> No partition metadata for
topic kafka1_topic2 due to kafka.common.LeaderNotAvailableException}]
for topic [kafka1_topic2]: class
kafka.common.LeaderNotAvailableException
(kafka.producer.BrokerPartitionInfo) --------------------------------------------------------------- Details are in this link -> https://community.hortonworks.com/questions/68641/kerberized-hdp-24-kafka-giving-error-kafkacommonle.html#answer-68645 Couple of days back, i'd got this issue & added the following constant to fix the issue - advertised.host.name=sandbox.hortonworks.com host.name=sandbox.hortonworks.com But the issue has re-occurred (above fix still in place), and not sure how to fix this. Any ideas on this ?
... View more
11-27-2016
07:06 PM
@Ayub Khan, @emaxwell - any inputs on this ?
... View more
11-27-2016
06:58 PM
Hi Kafka, HDP experts, I've a Kerberized HDP 2.4 sandbox.. i'm able to do kinit for the principal user (kafka), and publish & subscribe to topic. How do i do the same with additional users (e.g. kafka1) ? I've a Unix user - kafka1 setup.. How do i obtain a kinit for kafka1 user, so i can publish/subscribe messages to Kafka topic ?
... View more
Labels:
- Labels:
-
Apache Kafka
11-27-2016
01:54 AM
found fix in this link -> https://community.hortonworks.com/questions/5811/install-of-hdp-fails-with-valueerror-invalid-liter.html
... View more
11-27-2016
01:45 AM
Hi All, I'm trying to install Kerberos on HDP 2.4, and running into issues while installing Kerberos Client on Ambari UI here is the error snippet : Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py", line 37, in <module>
AfterInstallHook().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 219, in execute
method(env)
File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py", line 31, in hook
setup_hdp_symlinks()
File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py", line 44, in setup_hdp_symlinks
hdp_select.select_all(version)
File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/hdp_select.py", line 122, in select_all
Execute(command, only_if = only_if_command)
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 158, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 121, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 238, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
result = function(command, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
tries=tries, try_sleep=try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 'ambari-sudo.sh /usr/bin/hdp-select set all `ambari-python-wrap /usr/bin/hdp-select versions | grep ^2.4.0.0-169 | tail -1`' returned 1. Traceback (most recent call last):
File "/usr/bin/hdp-select", line 379, in <module>
printVersions()
File "/usr/bin/hdp-select", line 236, in printVersions
result[tuple(map(int, versionRegex.split(f)))] = f
ValueError: invalid literal for int() with base 10: 'kafka'
ERROR: set command takes 2 parameters, instead of 1 I've installed MIT KDC, and created the Principal, i get the error when enabling Kerberos using Ambari. Pls let me know if you have faced this issue. Thanks!
... View more
Labels:
11-26-2016
09:35 PM
@cmcbugg, pls see details of the issue in link -> https://community.hortonworks.com/questions/68497/kafka-error-while-fetching-metadata-topicmetadata.html Essentially, this is a fresh HDP 2.4 instance, and i've just enabled Ranger-Kafka plugin. Zookeper nodes permission : [kafka1@sandbox ~]$ ls -lrt /hadoop/zookeeper/ total 8 -rw-r--r-- 1
root root 1 2016-03-14 14:17 myid drwxr-xr-x 2 zookeeper hadoop 4096
2016-11-26 19:44 version-2 kafka1 User permission (on hdfs) : [kafka1@sandbox ~]$ hadoop fs -ls /user/ Found 11 items drwxrwx--- -
ambari-qa hdfs 0 2016-03-14 14:18 /user/ambari-qa drwxr-xr-x - hcat hdfs
0 2016-03-14 14:23 /user/hcat drwxr-xr-x - hive hdfs 0 2016-03-14 14:23
/user/hive drwxr-xr-x - kafka1 hdfs 0 2016-11-26 20:31 /user/kafka1
drwxr-xr-x - kafka2 hdfs 0 2016-11-26 20:32 /user/kafka2 Any ideas on what needs to be changed, to enable this ?
... View more
11-26-2016
05:45 PM
@Muhammad Syamsudin, i'm getting the same issue.. did you fix the issue ? Pls let me know the steps.
... View more
11-20-2016
11:47 PM
@Mohan V, @Ayub Khan - i'm facing the same issue, i'm on HDP 2.4 Is the issue resolved ?
... View more
11-20-2016
11:46 PM
@Shihab, how did you solve this issue ? what clean-up is required so kafka does not need authorization ? i'm getting the same issue .. I'm on HDP 2.4. thanks for your help in advance.
... View more
11-20-2016
09:45 PM
@eddie johnson, i changed the entry in /etc/resolv.conf file to -> 192.168.1.1, and it is able to access the external websites now ! Thanks for your help on this.
... View more