Yes, I add it in my execute command, but that does't work. ... View more

OK, I will share kafka state-change.log and please attention to test-kafka-topic at about 2017-07-12 16:04:00. Kafka state change log is state-change.txt ... View more

Hello, attachment is the operations in my environment. After I delete a topic named 'test-kafka-topic' I try to create a topic named 'test-kafka-topic', but it is created failed. And of course I set 'delete.topic.enable=true'. ... View more

My kafka server.properties content is # Generated by Apache Ambari. Tue May 16 06:15:33 2017 advertised.host.name=172.21.9.35 advertised.listeners=PLAINTEXT://172.21.9.35:6666,SSL://172.21.9.35:6667 allow.everyone.if.no.acl.found=true authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer auto.create.topics.enable=true auto.leader.rebalance.enable=true compression.type=producer controlled.shutdown.enable=true controlled.shutdown.max.retries=3 controlled.shutdown.retry.backoff.ms=5000 controller.message.queue.size=10 controller.socket.timeout.ms=30000 default.replication.factor=1 delete.topic.enable=false external.kafka.metrics.exclude.prefix=kafka.network.RequestMetrics,kafka.server.DelayedOperationPurgatory,kafka.server.BrokerTopicMetrics.BytesRejectedPerSec external.kafka.metrics.include.prefix=kafka.network.RequestMetrics.ResponseQueueTimeMs.request.OffsetCommit.98percentile,kafka.network.RequestMetrics.ResponseQueueTimeMs.request.Offsets.95percentile,kafka.network.RequestMetrics.ResponseSendTimeMs.request.Fetch.95percentile,kafka.network.RequestMetrics.RequestsPerSec.request fetch.purgatory.purge.interval.requests=10000 host.name=172.21.9.35 kafka.ganglia.metrics.group=kafka kafka.ganglia.metrics.host=localhost kafka.ganglia.metrics.port=8671 kafka.ganglia.metrics.reporter.enabled=true kafka.metrics.reporters=org.apache.hadoop.metrics2.sink.kafka.KafkaTimelineMetricsReporter kafka.timeline.metrics.host=node3 kafka.timeline.metrics.maxRowCacheSize=10000 kafka.timeline.metrics.port=6188 kafka.timeline.metrics.protocol=http kafka.timeline.metrics.reporter.enabled=true kafka.timeline.metrics.reporter.sendInterval=5900 kafka.timeline.metrics.truststore.password=bigdata kafka.timeline.metrics.truststore.path=/etc/security/clientKeys/all.jks kafka.timeline.metrics.truststore.type=jks leader.imbalance.check.interval.seconds=300 leader.imbalance.per.broker.percentage=10 listeners=PLAINTEXT://node1:6666,SSL://node1:6667 log.cleanup.interval.mins=10 log.dirs=/kafka-logs log.index.interval.bytes=4096 log.index.size.max.bytes=10485760 log.retention.bytes=-1 log.retention.hours=168 log.roll.hours=168 log.segment.bytes=1073741824 message.max.bytes=1000000 min.insync.replicas=1 num.io.threads=8 num.network.threads=3 num.partitions=1 num.recovery.threads.per.data.dir=1 num.replica.fetchers=1 offset.metadata.max.bytes=4096 offsets.commit.required.acks=-1 offsets.commit.timeout.ms=5000 offsets.load.buffer.size=5242880 offsets.retention.check.interval.ms=600000 offsets.retention.minutes=86400000 offsets.topic.compression.codec=0 offsets.topic.num.partitions=50 offsets.topic.replication.factor=3 offsets.topic.segment.bytes=104857600 port=6667 principal.builder.class=org.apache.kafka.common.security.auth.DefaultPrincipalBuilder producer.purgatory.purge.interval.requests=10000 queued.max.requests=500 replica.fetch.max.bytes=1048576 replica.fetch.min.bytes=1 replica.fetch.wait.max.ms=500 replica.high.watermark.checkpoint.interval.ms=5000 replica.lag.max.messages=4000 replica.lag.time.max.ms=10000 replica.socket.receive.buffer.bytes=65536 replica.socket.timeout.ms=30000 security.inter.broker.protocol=SSL socket.receive.buffer.bytes=102400 socket.request.max.bytes=104857600 socket.send.buffer.bytes=102400 ssl.client.auth=required ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1 ssl.key.password=hadoop ssl.keystore.location=/etc/kafka/conf/ssl/kafka.server.keystore.jks ssl.keystore.password=hadoop ssl.keystore.type=JKS ssl.truststore.location=/etc/kafka/conf/ssl/kafka.server.truststore.jks ssl.truststore.password=hadoop ssl.truststore.type=JKS super.users=User:CN=node1,OU=test,O=test,L=test,ST=test,C=te zookeeper.connect=node2:2181,node3:2181,node4:2181 zookeeper.connection.timeout.ms=25000 zookeeper.session.timeout.ms=30000 zookeeper.sync.time.ms=2000 ... View more

Yes, I set "security.inter.broker.protocol" as "SSL". ... View more

Hello, My produce command is $KAFKA_HOME/bin/kafka-console-producer.sh --broker-list node3:6667 --topic my-replicated-topic --producer.config /etc/kafka/conf/ssl/producer.properties consume command is ./bin/kafka-console-consumer.sh --bootstrap-server node1:6667 --topic my-replicated-topic --new-consumer --consumer.config /etc/kafka/conf/ssl/producer.properties --from-beginning I use SSL to authorize not Kerberos, so I add --security-protol SSL option to my Kafka Producer/Consumer. But in produce process, console prints messages as follows [2017-05-16 16:03:21,447] WARN Error while fetching metadata with correlation id 0 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:21,548] WARN Error while fetching metadata with correlation id 1 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:21,655] WARN Error while fetching metadata with correlation id 2 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:21,765] WARN Error while fetching metadata with correlation id 3 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:21,874] WARN Error while fetching metadata with correlation id 4 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:21,984] WARN Error while fetching metadata with correlation id 5 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:22,093] WARN Error while fetching metadata with correlation id 6 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:22,202] WARN Error while fetching metadata with correlation id 7 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:22,313] WARN Error while fetching metadata with correlation id 8 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:22,423] WARN Error while fetching metadata with correlation id 9 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:22,532] WARN Error while fetching metadata with correlation id 10 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:22,642] WARN Error while fetching metadata with correlation id 11 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:22,751] WARN Error while fetching metadata with correlation id 12 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:22,858] WARN Error while fetching metadata with correlation id 13 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:22,967] WARN Error while fetching metadata with correlation id 14 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:23,076] WARN Error while fetching metadata with correlation id 15 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:23,183] WARN Error while fetching metadata with correlation id 16 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:23,292] WARN Error while fetching metadata with correlation id 17 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:23,400] WARN Error while fetching metadata with correlation id 18 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:23,512] WARN Error while fetching metadata with correlation id 19 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:23,629] WARN Error while fetching metadata with correlation id 20 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:23,738] WARN Error while fetching metadata with correlation id 21 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) [2017-05-16 16:03:23,848] WARN Error while fetching metadata with correlation id 22 : {my-replicated-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) ... View more

Hi, does it mean that ranger kafka plugin can not define policy among users, and only among hosts? ... View more

yes, I can see in '/etc/ranger/mytest_hadoop/policycache' path there is a file named 'hdfs_mytest_hadoop.json'. the file contains policies under service 'mytest_hadoop'. ... View more

Hi, I can use my allow user to cat the assigned directory. Such as I want to use user 'hive' to cat '/user/accumulo' after I set allowing permission in ranger policy, but when I execute the command, hdfs permission policy prevent it. ... View more

OK. hdfs-ranger-audit-bigdata001istuarycom.txt ... View more

Yes, I add before. But it can not affect. And I see xa_portal.log not change. ... View more

I find pom.xml in contrib/views/tez path that "tez.view.version" is 0.7.0.0-SNAPSHOT, but in stack_advisor.py there is such code: if os.path.exists(views_work_dir) and os.path.isdir(views_work_dir): last_version = '0.0.0' for file in os.listdir(views_work_dir): if fnmatch.fnmatch(file, 'TEZ{*}'): current_version = file.lstrip("TEZ{").rstrip("}") # E.g.: TEZ{0.7.0.2.3.0.0-2154} if self.versionCompare(current_version.replace("-", "."), last_version.replace("-", ".")) >= 0: latest_tez_jar_version = current_version last_version = current_version pass pass pass<br> def versionCompare(self, version1, version2): def normalize(v): return [int(x) for x in re.sub(r'(\.0+)*$','', v).split(".")] return cmp(normalize(version1), normalize(version2)) pass So when versionCompare function is called, 'SNAPSHOT' can not transformed to int, so throw exception. But ambari does not change this, and it can run. In my environment, there is error. ... View more

No, I custom a stack named isdp, so under my stack directory as follows: ... View more

Yes, you are right. Ambari-server log is ... View more

First, I Enable Ranger Plugin for HDFS, and then save the change, but there is an error message "The configuration changes could not be validated for consistency due to an unknown error. Your changes have not been saved yet. Would you like to proceed and save the changes?". I do not know why it happens the error message, can you help me? ... View more

The "/check" path must registered? ... View more

I delete the @Produces, but the error is same. ... View more

the path is not registered? ... View more