Member since
03-18-2014
26
Posts
3
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
25188 | 03-31-2014 07:33 PM |
04-01-2014
04:28 PM
You are correct. I only noticed this after the command. Should have restricted to the /tmp. I'm not concerned as this is a research environment with no restricted data. Thanks dlo, Carl
... View more
04-01-2014
01:09 PM
Thanks Dio, the only minor issue on hiveserver2 was that it kept trying to restart because of the following error: org.apache.hadoop.security.AccessControlException: Permission denied: user=hive, access=WRITE, inode="/tmp":hdfs:supergroup:drwxr-xr-x Which clearly is a permission issue quickly solved with a chmod of the hadoop file system [root@hadoopmngr tmp]# sudo -u hive hadoop fs -ls /tmp Found 2 items drwxr-xr-x - hdfs supergroup 0 2014-04-01 15:50 /tmp/.cloudera_health_monitoring_canary_files drwxrwxrwt - mapred hadoop 0 2014-03-31 22:20 /tmp/logs [root@hadoopmngr tmp]# sudo -u hdfs hadoop fs -chmod -R 777 / [root@hadoopmngr tmp]# sudo -u hive hadoop fs -ls /tmp Found 2 items drwxrwxrwx - hdfs supergroup 0 2014-04-01 15:55 /tmp/.cloudera_health_monitoring_canary_files drwxrwxrwt - mapred hadoop 0 2014-03-31 22:20 /tmp/logs And t hen the hive server2 started just fine. Thanks again for your help. Carl
... View more
03-31-2014
07:33 PM
3 Kudos
retrying.. after deleting everything in /dfs by: cd /dfs rm -r -f * on each host including the CDh manager server... This actually worked!!! thanks Darren, Thanks DIO.. here are the steps when you need to reinstall from a botched delete of hive or zookeeper and the service add gives you a null pointer exception, (see my prior posts). 1) Stop the cluster 2) delete the cluster 3) cd /dfs on each host 4) rm -r -f * 5) using the manager add the cluster 6) select the hosts tab and select alll hosts 7) choose all the defaults and click continue to start firstrun() results: Command Progress Completed 20 of 20 steps. Waiting for ZooKeeper Service to initialize Finished waiting Details Starting ZooKeeper Service Completed 1/1 steps successfully Details Checking if the name directories of the NameNode are empty. Formatting HDFS only if empty. Sucessfully formatted NameNode. Details Starting HDFS Service Successfully started HDFS service Details Creating HDFS /tmp directory HDFS directory /tmp already exists. Details Creating MR2 job history directory Successfully created MR2 job history directory. Details Creating NodeManager remote application log directory Successfully created NodeManager remote application log directory. Details Starting YARN (MR2 Included) Service Successfully started service Details Creating Hive Metastore Database Created Hive Metastore Database. Details Creating Hive Metastore Database Tables Created Hive Metastore Database Tables successfully. Details Creating Hive user directory Successfully created Hive user directory. Details Creating Hive warehouse directory Successfully created Hive warehouse directory. Details Starting Hive Service Service started successfully. Details Creating Oozie database Oozie database created successfully. Details Installing Oozie ShareLib in HDFS Successfully installed Oozie ShareLib Details Starting Oozie Service Service started successfully. Details Creating Sqoop user directory Successfully created Sqoop user directory. Details Starting Sqoop Service Service started successfully. Details Starting Hue Service Service started successfully. Details
... View more
03-31-2014
07:07 PM
okay.. progress... I've now gotten past the HDFS format by removing /dfs/nn on the manager but now I have 0 datanodes started because of the following error on each node: 10:03:06.781 PM FATAL org.apache.hadoop.hdfs.server.datanode.DataNode Initialization failed for block pool Block pool <registering> (Datanode Uuid unassigned) service to hadoopmngr/192.168.0.102:8022java.io.IOException: Incompatible clusterIDs in /dfs/dn: namenode clusterID = cluster111; datanode clusterID = cluster6
... View more
03-31-2014
06:40 PM
Basically DIO if you see what is happening, because the format HDFS does not complete, the hive database setup step is not executed either, then when we try to start hive, we get a postgresql access denied error. Progress Command Context Status Started at Ended at First Run Finished Mar 31, 2014 9:35:56 PM EDT Mar 31, 2014 9:36:55 PM EDT Failed to perform First Run of services. Command Progress Completed 3 of 20 steps. Waiting for ZooKeeper Service to initialize Finished waiting Details Starting ZooKeeper Service Completed 1/1 steps successfully Details Checking if the name directories of the NameNode are empty. Formatting HDFS only if empty. Command (563) has failed Details Starting HDFS Service Creating HDFS /tmp directory Creating MR2 job history directory Creating NodeManager remote application log directory Starting YARN (MR2 Included) Service Creating Hive Metastore Database Creating Hive Metastore Database Tables Creating Hive user directory Creating Hive warehouse directory Starting Hive Service Creating Oozie database Installing Oozie ShareLib in HDFS Starting Oozie Service Creating Sqoop user directory Starting Sqoop Service Starting Hue Service Deploying Client Configuration
... View more
03-31-2014
06:34 PM
Database Setup On this page you configure and test database connections. If using custom databases, create the databases first according to the Installing and Configuring an External Database section of the Installation Guide. Use Custom DatabasesUse Embedded Database When using the Embedded Database, passwords are auto generated. Please copy them down. Hive Database Host Name: Database Type: Database Name : Username: Password: hadoopmngr:7432 MySQLPostgreSQLOracle hive1 hive1 dAPyUZE0mt
... View more