Member since
08-08-2017
1652
Posts
30
Kudos Received
11
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1969 | 06-15-2020 05:23 AM | |
| 16038 | 01-30-2020 08:04 PM | |
| 2108 | 07-07-2019 09:06 PM | |
| 8245 | 01-27-2018 10:17 PM | |
| 4669 | 12-31-2017 10:12 PM |
12-03-2017
05:32 PM
we have ambari cluster version 2.6 , this ambari cluster configured with service auto start enabled for all the services in our cluster we stop all services as HDFS , YARN , Zookeeper . Hive , etc and reboot the masters machines ( master01,master02,master03 ) we suppose to get the services up after reboot , all machines linux are up include ambari-server and ambari-agent on all masters machines but all services are down include ZK , HDFS , YARN and all restt we not understand why the ambari feature - service auto start not start auto the services after boot ? how to know if service auto start is working here ? and how top trace it ? after one hour from the reboot still all services are down !
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hadoop
12-03-2017
02:26 PM
Wק can abort the deletion by running the following command in zookeeper: rmr /admin/delete_topics/gtom.poli.pri.procis Note that this will only prevent the deletion to proceed if it has not already started. If anything has already been deleted, it's gone
... View more
12-03-2017
08:53 AM
just in case we delete the wrong topic as /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper hdpmaster01:2181 --delete --topic gtom.poli.pri.proc
Topic gtom.poli.pri.procis already marked for deletion and now we want to revert it what are the steps to revert topic deletion? all folders topics as the following are still exists under /var/kafka/kafka-logs gtom.poli.pri.procis-23
gtom.poli.pri.procis-45
gtom.poli.pri.procis-34 . .
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hadoop
-
Apache Kafka
12-02-2017
11:04 PM
thank you for the answer , but we create another new worker machine instead that machine , I think it was waste of time find the problem on that machine and better to create a new one
... View more
12-02-2017
07:00 PM
@Jordan - regrading what you said purge the Zookeeper records : do you mean to delete the topic by rmr /brokers/topics/hgpo.llo.prmt.processed from the Zc server
... View more
12-02-2017
05:32 PM
another thing the topics are exist under /brokers/topics/* and not under /consumers/..../.. so in this case I gues we need to remove the topic as rmr /brokers/topics/hgpo.llo.prmt.processed , can you confirm this please
... View more
12-02-2017
04:13 PM
just to be sure , so it will be safe to remove the topics use rm-rf as ( rm -rf hgpo.llo.prmt.processed-28 . rm -rf hgpo.llo.prmt.processed-28 and so on on all nodes ) ?
... View more
12-01-2017
11:17 AM
we do the following steps in order to delete the topic - hgpo.llo.prmt.processed but even after 12 hours topics folders still not deleted from /var/kafka/kafka-logs note - we set - delete.topic.enable=true hkafka01 kafka-logs]# /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper master01:2181 --alter --topic hgpo.llo.prmt.processed--config retention.ms=1000
WARNING: Altering topic configuration from this script has been deprecated and may be removed in future releases. Going forward, please use kafka-configs.sh for this functionality Updated config for topic "hgpo.llo.prmt.processedd"
kafka01 kafka-logs]# /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper master01:2181 --delete --topic hgpo.llo.prmt.processed
Topic hgpo.llo.prmt.processed is already marked for deletion. how to force the topic deletion ? on kafka machine ( production machine ) , we see that: ( no free space ) /dev/sdb 15500450528 15500434144 0 100% /var/kafka and under /var/kafka/kafka-logs , we still see all topic folders (huge size) as example: ( down ) so is it possible just to remove these folders as example - by rm -rf hgpo.llo.prmt.processed-28 ? under /var/kafka/kafka-logs we have many topic folders as : 117G hgpo.llo.prmt.processed-28
117G hgpo.llo.prmt.processed-29
117G hgpo.llo.prmt.processed-3
117G hgpo.llo.prmt.processed-30
117G hgpo.llo.prmt.processed-31
117G hgpo.llo.prmt.processed-32
.
.
.
.
.
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hadoop
-
Apache Kafka
12-01-2017
08:31 AM
@Venkata thank you , I run the script with the topic-name (hgpo.llo.prmt.processed) instead test , and consumer-group not found , I mean no output from the script , is is possible ? second as you know we get Topic mcapi.avro.pri.processed is already marked for deletion after we run "delete kafka topic" and we run it yesterday - why it take so along time ? , ( in my kafka - delete.topic.enable=true is set by default) so in that case can we remove the topic under /var/kafka/kafka-logs by rm -rf "topic name " ? as rm -rf hgpo.llo.prmt.processed-28 rm -rf hgpo.llo.prmt.processed-29 and so on for all other topics
117G hgpo.llo.prmt.processed-3 117G hgpo.llo.prmt.processed-30 117G hgpo.llo.prmt.processed-31 117G hgpo.llo.prmt.processed-32 . . .
... View more
11-30-2017
09:36 PM
just one note , when I delete a topic we get - Topic <....> is marked for deletion. , is it mean that it will take time until topic will be deleted ?
... View more