About athtsang

athtsang · ‎05-27-2015

I tried to use a script which start / stop CM agent. It's found that if the agent was started and then stopped, the script will be killed. As show below: [root@cdh52 ~]# cat myscript.sh #!/bin/bash service cloudera-scm-agent start service cloudera-scm-agent stop echo "Still Alive" You have new mail in /var/spool/mail/root [root@cdh52 ~]# ./myscript.sh Starting cloudera-scm-agent: [ OK ] Stopping cloudera-scm-agent: Terminated Note that [OK] and "Still Alive" is not displayed, because the parent script was killed when cloudera-scm-agent trying to stop the agent. The same effect if "service cloudera-scm-agent" was replaced by /etc/init.d/cloudera-scm-agent What i don't understand is HOW kill find the parent processto kill. The cloudera-scm-agent script has already disown'ed the agent. And ps show that the PID of agent's parent is 1 (init), not myscript.sh. Any suggestion?

athtsang · ‎05-19-2015

On a HA HDFS, I tried to add SecondaryNameNode. Here is what I face: When I add the SecondaryNameNode role instance, I got "validation error": Federated SecondaryNameNode secondarynamenode (mybigdata) is not configured with a Nameservice When I configure nameservice as the nameservice of the HDFS cluster, I got "validation error" : Nameservice nameservice1 has 2 NameNodes and 1 SecondaryNameNode(s) So, how to correctly configure the SecondaryNameNode? What's the correct number of NameNode / SecondaryNameNode Actually, we are getting around 200MB edit log in 1 month. Is is OK not to have SecondaryNameNode? CDH 5.1, 5.2

athtsang · ‎04-01-2015

Eventually, I extracted the CM Daemons RPM (using rpm2cpio and cpio), compared the files, and copied the missing files. CM is working again.

athtsang · ‎04-01-2015

Cloudera Manager 5.1.2 + CentOS 6 After disk failure of /usr partition, some CM installation files were lost (no backup ...). Any suggestion on how to repair the CM installation? Only /usr was affected, therefore, configuration info in /etc/ and /var should still be OK (That's why I still hope to repair the installation) . Will extracting the RPM to replace corrupted files work?

athtsang · ‎03-17-2015

Not sure whether all these are correct. But I managed to make all instance "Green" with some manual operation... JounalNode: I stopped another (good) JournalNode and copied the 'jn/[nameservice]/current' directory to the bad JournalNode. I tried copying while the good JournalNode was running, but starting the bad JournalNode gave same error as previously. NameNode: I copied the fsimage file mentioned by the log from the a good NameNode.

athtsang · ‎03-17-2015

CDH 5.1.3 installed with Parcel, HDFS HA enabled. After a server crash (node running NameNode in standby mode and JournalNode), issues occur during restart. In NameNode log: Failed to load image from FSImageFile(file=/data1/dfs/nn/current/fsimage_0000000000004637167, cpktTxId=0000000000004637167) java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:194) at org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:221) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:906) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:892) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:715) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:653) at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:276) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:882) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:629) at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:498) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:554) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:720) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:704) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1354) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1420) In JournalNode log: IPC Server handler 3 on 8485, call org.apache.hadoop.hdfs.qjournal.protocol.QJournalProtocol.getEditLogManifest from 192.168.88.37:53375 Call#2 Retry#0: output error How can I recover the node? Thanks

athtsang · ‎02-22-2015

This is by design. I forgot the rationale. It can be avoided by setting env variable TERM to linux (for bash and friends) export TERM=linux

athtsang · ‎02-15-2015

hour(timestamp) returns an INT, but trunc(timestamp, ...) returns a timestamp. So only trunc() can use with seconds_add(). If you have from_unixtime() and unix_timestamp(), you can apply the same concept (this seems even cleaner): select now(), concat( cast(from_unixtime( unix_timestamp(now()) - unix_timestamp(now()) % 900) as string), ' to ', cast(from_unixtime(899 + unix_timestamp(now()) - unix_timestamp(now()) % 900) as string) ) timeslot

athtsang · ‎02-13-2015

Clumsy but works, actually seems quite efficiently select now(), concat( cast(seconds_add(trunc(now(), 'hh'), (extract(now(), 'MINUTE') - extract(now(), 'MINUTE') % 15) * 60) as string), ' to ', cast(seconds_add(trunc(now(), 'hh'), (extract(now(), 'MINUTE') - extract(now(), 'MINUTE') % 15) * 60 + 899) as string)) timeslot;

athtsang · ‎11-05-2014

As a follow up, I configured Hadoop Security within Cloudera Manager following http://www.cloudera.com/content/cloudera/en/documentation/cloudera-manager/v5-0-0/Configuring-Hadoop-Security-with-Cloudera-Manager/cm5chs_using_cm_sec_config.html Things seems OK. The biggest problem I faced when configuring Security in CM was I attempted a non-default realm. This triggered a bug and so that the configuration failed, as speciifed in step 6. I found no such warning in http://www.cloudera.com/content/cloudera/en/documentation/cdh5/v5-0-0/CDH5-Security-Guide/CDH5-Security-Guide.html (or I overlooked). Wondering whether this caused my original failure.

Online	Offline
Last Visited	‎07-03-2019 10:01 PM

Member Since	‎11-03-2014 11:01 PM
Last Visited	‎07-03-2019 10:01 PM
Posts	46
Kudos received	8

Cloudera Community

Re: Fixing Over-replicated Blocks

Re: Isolation between Flume Channels?

Re: Problem Controlling CM Agent Start / Stop in a...

Re: Repairing a corrupt Cloudera Manager Installat...

Re: After server crash, HA Standby NameNode "Prema...

Problem Controlling CM Agent Start / Stop in a Scr...

Nameservice nameservice1 has 2 NameNodes and 1 Sec...

Re: Repairing a corrupt Cloudera Manager Installat...

Repairing a corrupt Cloudera Manager Installation

Re: After server crash, HA Standby NameNode "Prema...

After server crash, HA Standby NameNode "Premature...

Re: Funny characters in output file from Impala sh...

Re: 15 minuutes Time slot Bucket in Impala

Re: 15 minuutes Time slot Bucket in Impala

Re: "TSocket read 0 bytes" using Hue Impala Kerber...