About lenu

lenu · ‎10-29-2018

Hello everybody, My disk space is consuming more and more in HDFS and when I dig , I can see HBASE took 400GB of Old Wals .However If I look at Wals . it is almost empty . So Can I remove the complete OLDWals from the machine . Does it affect any data. I have read WALs are used only when there is crash in memory before writing it to disk . Still do not want to take the risk . $ hdfs dfs -du -h /apps/hbase/data 0/apps/hbase/data/.tmp ©37 0/apps/hbase/data/MasterProcWALs 12.6 K /apps/hbase/data/WALs ---- ONLY KBs 7.5 G /apps/hbase/data/archive 0/apps/hbase/data/corrupt 52.9 T /apps/hbase/data/data 42/apps/hbase/data/hbase.id 7/apps/hbase/data/hbase.version 379.5 G /apps/hbase/data/oldWALs -- Large data My Question is if I set any parameter in Ambari to auto purge on regular interval and how can I check the replication is enabled or not . hbase.master.logcleaner.ttl where to set if in case i do not want to delete it .

lenu · ‎10-28-2018

Simple and Cool. However the table is updated every other hour . It is taking very long time for 900GB to CTAS. The thing is to store TB of data for the first time and then 100GB daily incremental like insert/update/delete in HBase and to make it available for Business analysts . It is taking more than 40+ minutes to retrieve a single query. Loading the data in HBASE takes only 10 to 20 minutes.Any other approach Shu.Kindly give me some spark

lenu · ‎10-28-2018

@Shu Do you have steps to export the HBASE to Hive ORC table. I have tried the performance tuning already it didnt come up properly. Thank you very much helping

lenu · ‎10-28-2018

Hello Everybody, We have HBASE Table with around 10 Million records and when we integrate with Hive . It is taking more than 30 minutes to produce the results . If we try to do it in HBase it is fast. Is there anyway to manage the situation . Or 1.can I export all the data from HBase to Hive 2.How can we avoid full scan in HBase tables from Hive . Sorry for the basic questions

lenu · ‎10-12-2018

This post is currently awaiting moderation. If you believe this to be in error, contact a system administrator. https://community.hortonworks.com/questions/225074/nifi-to-write-data-into-hdfs-as-different-user.html

lenu · ‎10-04-2018

Hi All, Is there any way to find who has restarted the services through Ambari .In our team six people working on different shifts and teams . It is very difficult to track who has initiated the service restart . For example . Some one restarted Namenode and we were unable to find who has initiated the restart via ambari . Attached the ideal image of Ambari services restart . 10377-timedout.png We have searched the logs with all 6 userid . The log returned nothing with the userids . Is there any alternative ways or how we can easily identify who has done the restart from Ambari .

lenu · ‎10-03-2018

Hi All, We have 10 node cluster. We have only few teams are using the cluster at the moment.What do you suggest fresh install or Upgrade . If so why ? Could you please explain the pain points of both . What is the best practice here . Clean installation of OS, HDP , HDF or upgrade of HDP and HDF . If fresh install . we will take a back of all the data to another machine and reinstall everything .

Online	Offline
Last Visited	‎12-26-2018 12:29 PM

Member Since	‎09-18-2018 05:33 AM
Last Visited	‎12-26-2018 12:29 PM
Posts	13

Cloudera Community

HBASE OldWals not deleting

Re: Underlying HBASE Table is taking 30+ minutes f...

Re: Underlying HBASE Table is taking 30+ minutes f...

Underlying HBASE Table is taking 30+ minutes for s...

Cannot post questions

Find the user who restarted any services

Upgrade or Fresh install