Member since
11-03-2014
46
Posts
8
Kudos Received
7
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
5985 | 11-01-2016 11:02 PM | |
2019 | 01-05-2016 07:27 PM | |
3592 | 05-27-2015 09:15 PM | |
1639 | 04-01-2015 11:39 PM | |
3629 | 03-17-2015 09:31 PM |
05-27-2015
04:08 AM
I tried to use a script which start / stop CM agent. It's found that if the agent was started and then stopped, the script will be killed. As show below: [root@cdh52 ~]# cat myscript.sh
#!/bin/bash
service cloudera-scm-agent start
service cloudera-scm-agent stop
echo "Still Alive"
You have new mail in /var/spool/mail/root
[root@cdh52 ~]# ./myscript.sh
Starting cloudera-scm-agent: [ OK ]
Stopping cloudera-scm-agent: Terminated Note that [OK] and "Still Alive" is not displayed, because the parent script was killed when cloudera-scm-agent trying to stop the agent. The same effect if "service cloudera-scm-agent" was replaced by /etc/init.d/cloudera-scm-agent What i don't understand is HOW kill find the parent processto kill. The cloudera-scm-agent script has already disown'ed the agent. And ps show that the PID of agent's parent is 1 (init), not myscript.sh. Any suggestion?
... View more
Labels:
- Labels:
-
Cloudera Manager
05-19-2015
03:12 AM
On a HA HDFS, I tried to add SecondaryNameNode. Here is what I face: When I add the SecondaryNameNode role instance, I got "validation error": Federated SecondaryNameNode secondarynamenode (mybigdata) is not configured with a Nameservice When I configure nameservice as the nameservice of the HDFS cluster, I got "validation error" : Nameservice nameservice1 has 2 NameNodes and 1 SecondaryNameNode(s) So, how to correctly configure the SecondaryNameNode? What's the correct number of NameNode / SecondaryNameNode Actually, we are getting around 200MB edit log in 1 month. Is is OK not to have SecondaryNameNode? CDH 5.1, 5.2
... View more
Labels:
- Labels:
-
HDFS
04-01-2015
11:39 PM
Eventually, I extracted the CM Daemons RPM (using rpm2cpio and cpio), compared the files, and copied the missing files. CM is working again.
... View more
04-01-2015
03:18 AM
Cloudera Manager 5.1.2 + CentOS 6 After disk failure of /usr partition, some CM installation files were lost (no backup ...). Any suggestion on how to repair the CM installation? Only /usr was affected, therefore, configuration info in /etc/ and /var should still be OK (That's why I still hope to repair the installation) . Will extracting the RPM to replace corrupted files work?
... View more
Labels:
- Labels:
-
Cloudera Manager
03-17-2015
09:31 PM
1 Kudo
Not sure whether all these are correct. But I managed to make all instance "Green" with some manual operation... JounalNode: I stopped another (good) JournalNode and copied the 'jn/[nameservice]/current' directory to the bad JournalNode. I tried copying while the good JournalNode was running, but starting the bad JournalNode gave same error as previously. NameNode: I copied the fsimage file mentioned by the log from the a good NameNode.
... View more
03-17-2015
07:32 PM
CDH 5.1.3 installed with Parcel, HDFS HA enabled. After a server crash (node running NameNode in standby mode and JournalNode), issues occur during restart. In NameNode log: Failed to load image from FSImageFile(file=/data1/dfs/nn/current/fsimage_0000000000004637167, cpktTxId=0000000000004637167) java.io.IOException: Premature EOF from inputStream at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:194) at org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:221) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:906) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:892) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:715) at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:653) at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:276) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:882) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:629) at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:498) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:554) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:720) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:704) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1354) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1420) In JournalNode log: IPC Server handler 3 on 8485, call org.apache.hadoop.hdfs.qjournal.protocol.QJournalProtocol.getEditLogManifest from 192.168.88.37:53375 Call#2 Retry#0: output error How can I recover the node? Thanks
... View more
Labels:
- Labels:
-
HDFS
02-22-2015
06:25 PM
1 Kudo
This is by design. I forgot the rationale. It can be avoided by setting env variable TERM to linux (for bash and friends) export TERM=linux
... View more
02-15-2015
05:53 PM
1 Kudo
hour(timestamp) returns an INT, but trunc(timestamp, ...) returns a timestamp. So only trunc() can use with seconds_add(). If you have from_unixtime() and unix_timestamp(), you can apply the same concept (this seems even cleaner): select now(),
concat(
cast(from_unixtime(
unix_timestamp(now()) - unix_timestamp(now()) % 900) as string),
' to ',
cast(from_unixtime(899 +
unix_timestamp(now()) - unix_timestamp(now()) % 900) as string)
) timeslot
... View more
02-13-2015
12:14 AM
Clumsy but works, actually seems quite efficiently select now(),
concat(
cast(seconds_add(trunc(now(), 'hh'),
(extract(now(), 'MINUTE') - extract(now(), 'MINUTE') % 15) * 60) as string),
' to ',
cast(seconds_add(trunc(now(), 'hh'),
(extract(now(), 'MINUTE') - extract(now(), 'MINUTE') % 15) * 60 + 899) as string)) timeslot;
... View more
11-05-2014
02:57 AM
As a follow up, I configured Hadoop Security within Cloudera Manager following http://www.cloudera.com/content/cloudera/en/documentation/cloudera-manager/v5-0-0/Configuring-Hadoop-Security-with-Cloudera-Manager/cm5chs_using_cm_sec_config.html Things seems OK. The biggest problem I faced when configuring Security in CM was I attempted a non-default realm. This triggered a bug and so that the configuration failed, as speciifed in step 6. I found no such warning in http://www.cloudera.com/content/cloudera/en/documentation/cdh5/v5-0-0/CDH5-Security-Guide/CDH5-Security-Guide.html (or I overlooked). Wondering whether this caused my original failure.
... View more
- « Previous
- Next »