is it becoming so annoying since the difference now between the UI ( or hdfs dfsadmin -report) and hdfs dfs -du -h -s now 150T, i delete all the hdfs snapshots and disallow them but still get the same results. ... View more

@sunilosunil Are you using cloudera manager:   Authentication Backend desktop.auth.backend.LdapBackend LDAP URL ldap://your_ldap_url LDAP Search Base LDAP Bind User LDAP Bind Password LDAP User Filter LDAP Username Attribute LDAP Group Filter LDAP Group Name Attribute LDAP Group Membership Attribute Active Directory Domain   You need your system admin to create you a user in the LDAP and provide you with this parameters.   Then you can just restart Hue service ... View more

Don't know the exact cause that it was not working even i restarted the Yarn, when i had another chance to restart the Yarn, the issue was resolved. ... View more

@Harsh J For my special case where the hadoop nodes uptime was 1200 days and the servers with old centos versions, restarted the servers took the inodes down from 88% to 10%. ... View more

Hi Harsh, This is the drive formatting for this cross all the farms. We were fine with these for years. Still wondering if such inode definition for 3 nodes is enough for 700 K objects. I deleted the jobcache and usercache and delete un needed files and drop the objects from 900 K to 700 K but insides still 82%. This makes me wondering what else other than the HDFS file can affect this. ... View more

Is that mean i need to broke all my production servers and give it special treatment by enabling stepping to get Kudu working for me?   Seems we are so far to go with it to our production, i'm using it in our test enviroment. ... View more

i already restarted them as the process of adding the spark2 procedure. I will give it another restart and check ... View more

Seems the spark2 service isn't shown for me at cloudera manager, even the parcels are distributed What i'm missing? see below: [root@aopr-dhc001 parcels]# ll total 20 lrwxrwxrwx 1 cloudera-scm cloudera-scm 27 Jul 25 00:52 CDH -> CDH-5.10.2-1.cdh5.10.2.p0.5 drwxr-xr-x 11 cloudera-scm cloudera-scm 4096 Jun 27 01:39 CDH-5.10.2-1.cdh5.10.2.p0.5 lrwxrwxrwx 1 cloudera-scm cloudera-scm 33 Jul 25 00:52 GPLEXTRAS -> GPLEXTRAS-5.10.2-1.cdh5.10.2.p0.5 drwxr-xr-x 4 cloudera-scm cloudera-scm 4096 Jun 27 01:52 GPLEXTRAS-5.10.2-1.cdh5.10.2.p0.5 drwxr-xr-x 9 cloudera-scm cloudera-scm 4096 Nov 22 2016 IMPALA_KUDU-2.7.0-3.cdh5.9.0.p0.10 lrwxrwxrwx 1 cloudera-scm cloudera-scm 28 Jul 25 00:54 KUDU -> KUDU-1.3.0-1.cdh5.11.0.p0.12 drwxr-xr-x 9 cloudera-scm cloudera-scm 4096 Apr 12 17:37 KUDU-1.3.0-1.cdh5.11.0.p0.12 lrwxrwxrwx 1 cloudera-scm cloudera-scm 43 Jul 25 01:01 SPARK2 -> SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904 drwxr-xr-x 6 cloudera-scm cloudera-scm 4096 Mar 29 02:57 SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904 ===================================================================== [root@aovr-cmc101 ~]# cd /opt/cloudera/csd/ [root@aovr-cmc101 csd]# ll total 36 -rwxrwxrwx 1 fawzea noc 19899 Nov 7 2016 KUDU-1.0.0.jar -rw-r--r-- 1 cloudera-scm cloudera-scm 16109 Jul 25 00:43 SPARK2_ON_YARN-2.1.0.cloudera1.jar ... View more

something going wrong and i'm may missed it.   The service i added to the cluster is called spark (Standalone), i don't see it as spark2 on Yarn,   Also when i navigate the history server it show me the version is spark 1.6 ... View more