Hello All,   I went to the mysql command line and used schema create/update  -  able to get the metastore working & also able to get the tables form hive as well as impala.   database :mysql  datastore database: hive    mysql> source hive-schema-0.13.0.mysql.sql; mysql> use hive Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> source upgrade-0.10.0-to-0.11.0.mysql.sql +--------------------------------------------------+ | | +--------------------------------------------------+ | Upgrading MetaStore schema from 0.10.0 to 0.11.0 | +--------------------------------------------------+ 1 row in set, 1 warning (0.00 sec) +-----------------------------------------------------------+ | | +-----------------------------------------------------------+ | Finished upgrading MetaStore schema from 0.10.0 to 0.11.0 | +-----------------------------------------------------------+ 1 row in set, 1 warning (0.00 sec) mysql> source upgrade-0.11.0-to-0.12.0.mysql.sql +--------------------------------------------------+ | | +--------------------------------------------------+ | Upgrading MetaStore schema from 0.11.0 to 0.12.0 | +--------------------------------------------------+ 1 row in set, 1 warning (0.00 sec) +---------------------------------------------------+ | | +---------------------------------------------------+ | < HIVE-3255 Master Key and Delegation Token DDL > | +---------------------------------------------------+ 1 row in set, 1 warning (0.00 sec) Query OK, 0 rows affected, 1 warning (0.00 sec) Query OK, 0 rows affected, 1 warning (0.00 sec) +-----------------------------------------------------------+ | | +-----------------------------------------------------------+ | < HIVE-3764 Support metastore version consistency check > | +-----------------------------------------------------------+ 1 row in set, 1 warning (0.00 sec) Query OK, 0 rows affected, 1 warning (0.00 sec) ERROR 1062 (23000): Duplicate entry '1' for key 1 Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0 +-----------------------------------------------------------+ | | +-----------------------------------------------------------+ | Finished upgrading MetaStore schema from 0.11.0 to 0.12.0 | +-----------------------------------------------------------+ 1 row in set, 1 warning (0.00 sec) mysql> source upgrade-0.12.0-to-0.13.0.mysql.sql +--------------------------------------------------+ | | +--------------------------------------------------+ | Upgrading MetaStore schema from 0.12.0 to 0.13.0 | +--------------------------------------------------+ 1 row in set, 1 warning (0.00 sec) +-----------------------------------------------------------------------+ | | +-----------------------------------------------------------------------+ | < HIVE-5700 enforce single date format for partition column storage > | +-----------------------------------------------------------------------+ 1 row in set, 1 warning (0.00 sec) Query OK, 0 rows affected (0.00 sec) Rows matched: 0 Changed: 0 Warnings: 0 +--------------------------------------------+ | | +--------------------------------------------+ | < HIVE-6386: Add owner filed to database > | +--------------------------------------------+ 1 row in set, 1 warning (0.00 sec) ERROR 1060 (42S21): Duplicate column name 'OWNER_NAME' ERROR 1060 (42S21): Duplicate column name 'OWNER_TYPE' +---------------------------------------------------------------------------------------------+ | | +---------------------------------------------------------------------------------------------+ | <HIVE-6458 Add schema upgrade scripts for metastore changes related to permanent functions> | +---------------------------------------------------------------------------------------------+ 1 row in set, 1 warning (0.00 sec) Query OK, 0 rows affected, 1 warning (0.00 sec) Query OK, 0 rows affected, 1 warning (0.00 sec) +----------------------------------------------------------------------------------+ | | +----------------------------------------------------------------------------------+ | <HIVE-6757 Remove deprecated parquet classes from outside of org.apache package> | +----------------------------------------------------------------------------------+ 1 row in set, 1 warning (0.00 sec) Query OK, 0 rows affected (0.01 sec) Rows matched: 0 Changed: 0 Warnings: 0 Query OK, 0 rows affected (0.00 sec) Rows matched: 0 Changed: 0 Warnings: 0 Query OK, 0 rows affected (0.01 sec) Rows matched: 0 Changed: 0 Warnings: 0 ERROR 1050 (42S01): Table 'TXNS' already exists ERROR 1050 (42S01): Table 'TXN_COMPONENTS' already exists ERROR 1050 (42S01): Table 'COMPLETED_TXN_COMPONENTS' already exists ERROR 1050 (42S01): Table 'NEXT_TXN_ID' already exists Query OK, 1 row affected (0.00 sec) ERROR 1050 (42S01): Table 'HIVE_LOCKS' already exists ERROR 1061 (42000): Duplicate key name 'HL_TXNID_IDX' ERROR 1050 (42S01): Table 'NEXT_LOCK_ID' already exists Query OK, 1 row affected (0.00 sec) ERROR 1050 (42S01): Table 'COMPACTION_QUEUE' already exists ERROR 1050 (42S01): Table 'NEXT_COMPACTION_QUEUE_ID' already exists Query OK, 1 row affected (0.00 sec) Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0 +-----------------------------------------------------------+ | | +-----------------------------------------------------------+ | Finished upgrading MetaStore schema from 0.12.0 to 0.13.0 | +-----------------------------------------------------------+ 1 row in set, 1 warning (0.00 sec) mysql>     ... View more

Hello All,   This issue was resolved - working with Kevin Odel from  Cloudera - summary as follows   1.) First time install on Amazon AWS  - install done by using package 2.) Configured Kerberos 3.) All the services started good and nice 4.) We can move files in and out of hdfs - no issues 5.) YARN manager status will be good on  Cloudera Manager 6.) When you run example or pig scripts it will stuck at 0% with status of scheduled to submitted   Troubleshooting:   1.) go to YARN  Resource Manager Web UI            - click on Nodes                - you will not see any nodes listed (basically no node manager on the cluster)                - above is the reason you jobs stuck with 0%  2.) Go to each Node Manager look at /var/log/hadoop-yarn/*   You see the below error   2014-05-28 12:36:12,911 INFO org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Disk(s) failed. 1/1 local-dirs turned bad: /dfs/dn/yarn/nm; 2014-05-28 12:36:12,911 ERROR org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Most of the disks failed. 1/1 local-dirs turned bad: /dfs/dn/yarn/nm; 2014-05-28 16:29:50,916 INFO org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Disk(s) failed. 1/1 local-dirs turned bad: /dfs/dn/yarn/nm; 2014-05-28 16:29:50,916 ERROR org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Most of the disks failed. 1/1 local-dirs turned bad: /dfs/dn/yarn/nm; 2014-05-29 13:09:54,340 INFO org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Disk(s) failed. 1/1 local-dirs turned bad: /dfs/dn/yarn/nm; 2014-05-29 13:09:54,340 ERROR org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Most of the disks failed. 1/1 local-dirs turned bad: /dfs/dn/yarn/nm; 2014-06-02 13:43:10,736 INFO org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Disk(s) failed. 1/1 local-dirs turned bad: /dfs/dn/yarn/nm; 2014-06-02 13:43:10,736 ERROR org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Most of the disks failed. 1/1 local-dirs turned bad: /dfs/dn/yarn/nm; 2014-06-02 13:58:15,344 INFO org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Disk(s) failed. 1/1 local-dirs turned bad: /dfs/dn/yarn/nm; 2014-06-02 13:58:15,344 ERROR org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Most of the disks failed. 1/1 local-dirs turned bad: /dfs/dn/yarn/nm;   To Fix it:   go to CM  - YARN - make modification to yarn.local.dir   (make sure all the nodes on same group or do for each group of nodes) stop  YARN cluster deploy the configuration (with new yarn.local.dir) start  YARN Cluster now - go back to  YARN Resource Manager WEB UI  -you should see all the nodes       - if node is missing - it means it was not able to create a directory for local       - go to node which is not visiable on Resource manager and check at the /var/log/hadoop-yarn/* logs file       - fix manual or from CM      - redeploy configuration from CM      - start YARN services once you make sure all the nodes visiable on Resource Manager - Nodes on Yarn   You can submit your jobs  now - it will complete.   Thanks, Ram   ... View more

Hello All,   I worked with cloudera support to solve this issue. The issue is basically krb5cache file location not at c:\ProgramData\MIT\Kerberos5\   So you need to add the windows system variable  KRB5CCNAME and specify value as C:\ProgramData\MIT\Kerberos5\krb5cache  than - regenrate the ticket close the ODBC 32 administrator open ODBC 32 bit adaministrator test it now it will work   I have documented the full procedure needed to connect tableau 8.0.4 or higher to secure hive. So if you need it ..shoot me message.   Thanks, Ram ... View more

it was resolved   by issuing the following command kadmin.local: modprinc -maxrenewlife 7day krbtgt/YOUR_REALM modprinc -maxrenewlife 7day +allow_renewable impala/FQDN@YOUR_REALM ... View more