@Teradata GTS Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks. ... View more

@BORDIN Has the reply above helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks! ... View more

Hello All,   This is an older post which had a few recent followup queries. To close the loop, HBase offers multiple Tools to migrate Data from 1 Cluster to another Cluster like Snapshot, Export-Import, HashTable/SyncTable etc. Most of these Tools relies on MapReduce & uses 1 Mapper per Region of the Source Table. All these Tools works without any concerns. The only part of the ask which can't be answered accurately is the Concurrency/Job Configurations/Mapper Memory etc. These details rely on Customer's Environment Setup & the Bandwidth between the 2 Clusters. As such, Customer can run 1 such HBase MR Job & see the Outcome. Accordingly, Fine-Tune is required.    If any issues are observed while performing the above HBase MR Job, Feel free to post the Q in a Community Post for fellow Community Members to review & share their thoughts.    Regards, Smarak ... View more

@AkramSharkawy  If you are still experiencing the issue, can you provide the information @Elias has requested? Thanks ... View more

  @shrikantbm& team, Yes, in this case, we need to check cleanup.policy of the topic __consumer_offsets. If the existing cleanup.policy=compact then the log segment of this topic will not be deleted.   You should follow the below steps to conclude and resolve this issue initially.    1) Check what is current cleanup.policy of the topic __consumer_offsets. You can check it using the command:  kafka-topics.sh --bootstrap-server <broker-hostname:9092> --describe OR kafka-topics.sh --zookeeper <zookeeper-hostname:2181> --describe --topics-with-overrides   Note: topic_name is the name for which you are facing an issue   2)  If you want to clear the old log segment of this topic, then you should set cleanup.policy like cleanup.policy=compact,delete,retention.ms=<30days>    compact = when the kafka-log is rolled over, it will be compacted delete - once the offset.retention.ms is reached, the older logs will be removed retention.ms=<30days>  > the old log segment will be deleted after 30 days.   Note: 30 days are just an example here and this setting will be in ms. You should set it as per your requirement after checking it with the application team and their need.    For "delete", the property "log.cleaner.enable" must be set to "true"   After configuring this cleanup policy data will be deleted as per retention.ms as suggested above. If you will not set retention.ms then old log segment will be deleted as per retention period set in the CM / Ambari >> kafka >> Conf. The setting is log.retention.hours = <7 Days default> in CM >> Kafka, check what it is in your case so that log segment older than 7 days will be deleted. Kafka will keep checking the old log segment with the help of the property log.retention.check.interval.ms .   Important note: The "delete" on consumer offsets is that you may lose offsets which can lead to duplication/data loss. So check it with your application team before setting a deletion policy.   3) If you still face the same issue, then broker logs need to be reviewed for the root cause of the issue and make the changes accordingly.    If you found this information helped with your query, please take a moment to log in and click on KUDOS 🙂 and "Accept as Solution" below this post.   Thank you. ... View more

@Imran_chaush Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks ... View more

Hi Maggy,   Thanks for confirming my understanding regarding Classifications.  We are using Ranger with Atlas, however the 'most fine-grain' approach with Ranger is to use the Classifications as I understand it.   Would you know if any other approaches with Ranger would allow fine-grain security applying to Atlas Attributes? ... View more

@jeyaguna Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks ... View more

@Ploeplse Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks ... View more