I made sure that no jobs were running at that time when the deletion happened :). Also I am not sure who set retention period to 8 years, have to check that. One thing i want to know here, if before deletion, I have would have set the property to 7days and restarted only timeline server, would it remove all the older logs from leveldb and keep logs which are generated in last 7 days? I guess after restarting Yarn, Job history server also resets.. and gets clears at regular intervals.. not sure though!! ... View more

Thanks It worked. I just want to know if there are any consequences after deleting the leveldb-timeline-store.ldb directory. After deleteing content of this directory and restarting timeline server, the contents of this directory regenerated and freed up the space. We set “yarn.timeline-service.ttl-ms” from 267840000000(~443weeks) to 604800000(7 Days) to limit the size of the leveldb storage, according to hortonworks document. ... View more

Ambari version is 2.1.2 ... View more

Ambari version is 2.1.2 ... View more

when I do klist, It shows below output: klist /var/lib/ambari-agent/tmp/web_alert_cc_4610ec9b283dcfc90bc6df1e519e1c52 Ticket cache: FILE:/var/lib/ambari-agent/tmp/web_alert_cc_4610ec9b283dcfc90bc6df1e519e1c52 Default principal: HTTP/abcd_fqdn@REALM.COM Valid starting Expires Service principal 07/26/16 08:42:28 07/26/16 08:47:28 krbtgt/REALM.COM@REALM.COM When I run below command it just execute and didn't give any error in the output /usr/share/centrifydc/kerberos/bin/kinit -l 5m -c /var/lib/ambari-agent/tmp/web_alert_cc_4610ec9b283dcfc90bc6df1e519e1c52 -kt /etc/security/keytabs/spnego.service.keytab HTTP/abcd_fqdn.com@REALM.COM ... View more

How to confirm who manages Krb5.conf file. The ownership is with root: -rw-r--r-- 1 root root 727 May 11 17:00 krb5.conf ccache_type = 3 default_ccache_name is not defined Do I need to set this parameter for a particular UID or simply I give for all users default_ccache_name = /tmp/krb5cc_%{uid} and save conf file? ... View more

Is it a bug that shows port 8050 in Ambari UI but connect to port 8032..?? I still dont understand as why it doesn't connect to 8050 if we have set Yarn.resourcemanager.address to <rmaddress:8050> rather than connecting to 8032. ... View more

@Kuldeep Kulkarni , I tried this suggestion and it worked for me but oozie and spark jobs failed as they are not able to connect to current active RM. I had to manually switchover RM then restarted oozie and spark, then jobs started running. Weird though 🙂 Anyways do you have any document or KB Article related to your suggetsion where this is shown or described. Again, Thanks for your time 🙂 ... View more

Yes, I have RM HA ... View more

Then why it is connecting to port 8032 and not 8050 in our cluster. I have HDP2.3.4. Do we need to explicitly set yarn.resourcemanager.address.X to <IP.master1>:8050 and yarn.resourcemanager.address.Y to <IP.master2>:8050 in yarn-site.xml ... View more

gclog-data02.txtgclog-data06.txt ... View more

I dont see any RIT on HBase webui but I see read requests of 7M & 3M on few datanodes. Attaching logs. Is it advisable to restart ambari-server in this case ... View more

Thanks for the link. This issue is being faced by one of our teams and they have only access to Hue and not the command line. They are uploading archived zip files to HDFS and the size ranges from 2GB to 900GB. I already told them that Hue is not meant to upload such big files in HDFS and I am not able to find the background functionality of fileuploader anywhere online. ... View more

@Artem Ervits no networking or firewall issue as such ... View more

a) Datanodes are up and running. b) Will check the tcpdump and send the output ... View more

fsck output ......Status: HEALTHY Total size: 15311550475135 B (Total open files size: 36 B) Total dirs: 543341 Total files: 1572526 Total symlinks: 0 (Files currently being written: 5) Total blocks (validated): 1635306 (avg. block size 9363110 B) (Total open file blocks (not validated): 4) Minimally replicated blocks: 1635306 (99.99999 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 43091 (2.635042 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 3 Average block replication: 2.9589655 Corrupt blocks: 0 Missing replicas: 67060 (1.3669327 %) Number of data-nodes: 3 Number of racks: 1 ... View more

DFSadmin report Configured Capacity: 107909857935360 (98.14 TB) Present Capacity: 107907646763463 (98.14 TB) DFS Remaining: 63959036886116 (58.17 TB) DFS Used: 43948609877347 (39.97 TB) DFS Used%: 40.73% Under replicated blocks: 35016 Blocks with corrupt replicas: 113 Missing blocks: 0 Datanodes available: 3 (3 total, 0 dead) Live datanodes: Name: x.x.x.x4 Hostname: x.x.x.x4 Decommission Status : Normal Configured Capacity: 35969952645120 (32.71 TB) DFS Used: 15310914557283 (13.93 TB) Non DFS Used: 821825081 (783.75 MB) DFS Remaining: 20658216262756 (18.79 TB) DFS Used%: 42.57% DFS Remaining%: 57.43% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 14 Last contact: Wed Dec 16 20:17:25 AEDT 2015 Name: x.x.x.x6 Hostname: x.x.x.x6 Decommission Status : Normal Configured Capacity: 35969952645120 (32.71 TB) DFS Used: 14348334051328 (13.05 TB) Non DFS Used: 497512448 (474.46 MB) DFS Remaining: 21621121081344 (19.66 TB) DFS Used%: 39.89% DFS Remaining%: 60.11% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 1025 Last contact: Wed Dec 16 20:17:23 AEDT 2015 Name: x.x.x.x5 Hostname: x.x.x.x5 Decommission Status : Normal Configured Capacity: 35969952645120 (32.71 TB) DFS Used: 14289361268736 (13.00 TB) Non DFS Used: 891834368 (850.52 MB) DFS Remaining: 21679699542016 (19.72 TB) DFS Used%: 39.73% DFS Remaining%: 60.27% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 1025 ... View more

do you feel here any issues with ambari heap size?? ... View more

total used free shared buffers cached Mem: 251 2 249 0 0 0 -/+ buffers/cache: 1 250 Swap: 7 0 7 ... View more

Yes I have 256 GB box and only 10-15GB is occupied ... View more