Hi    manually installed krb5-server MIT and Workstation on the QUICK START VM .  I am using AES encryption type.  below is the error i am facing.    KRB.CONF  /ETC/KRB5.CONF default_tgs_enctypes = aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96 arcfour-hmac-md5 default_tkt_enctypes = aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96 arcfour-hmac-md5 permitted_enctypes = aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96 arcfour-hmac-md5 ---- Matches the exact sequence in Cloudera Manger also. edited /var/kerberos/krb5kdc/kadm5.acl with Then we edit /var/kerberos/krb5kdc/kdc.conf 1 [kdcdefaults] kdc_ports = 88 kdc_tcp_ports = 88 [realms] HADOOPSECURITY.COM = { #master_key_type = aes256-cts acl_file = /var/kerberos/krb5kdc/kadm5.acl dict_file = /usr/share/dict/words admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab supported_enctypes = aes256-cts-hmac-sha1-96:normal aes128-cts-hmac-sha1-96:normal des3-hmac-sha1:normal arcfour-hmac-md5:normal max_renewable_life = 7d } below is the error i am getting when i enable debug and stdout .    cloudera@quickstart Desktop]$ kinit hdfs Password for hdfs@HADOOPSEC.COM: [cloudera@quickstart Desktop]$ hdfs dfs -ls Java config name: null Native config name: /etc/krb5.conf Loaded from native config >>>KinitOptions cache name is /tmp/krb5cc_501 >>>DEBUG <CCacheInputStream> client principal is hdfs@HADOOPSEC.COM >>>DEBUG <CCacheInputStream> server principal is krbtgt/HADOOPSEC.COM@HADOOPSEC.COM >>>DEBUG <CCacheInputStream> key type: 18 >>>DEBUG <CCacheInputStream> auth time: Mon May 14 05:06:42 PDT 2018 >>>DEBUG <CCacheInputStream> start time: Mon May 14 05:06:42 PDT 2018 >>>DEBUG <CCacheInputStream> end time: Tue May 15 05:06:42 PDT 2018 >>>DEBUG <CCacheInputStream> renew_till time: Mon May 21 05:06:42 PDT 2018 >>> CCacheInputStream: readFlags() FORWARDABLE; RENEWABLE; INITIAL; >>>DEBUG <CCacheInputStream> client principal is hdfs@HADOOPSEC.COM >>>DEBUG <CCacheInputStream> server principal is X-CACHECONF:/krb5_ccache_conf_data/fast_avail/krbtgt/HADOOPSEC.COM@HADOOPSEC.COM >>>DEBUG <CCacheInputStream> key type: 0 >>>DEBUG <CCacheInputStream> auth time: Wed Dec 31 16:00:00 PST 1969 >>>DEBUG <CCacheInputStream> start time: null >>>DEBUG <CCacheInputStream> end time: Wed Dec 31 16:00:00 PST 1969 >>>DEBUG <CCacheInputStream> renew_till time: null >>> CCacheInputStream: readFlags() >>> unsupported key type found the default TGT: 18 18/05/14 05:06:52 WARN security.UserGroupInformation: PriviledgedActionException as:cloudera (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] 18/05/14 05:06:52 WARN ipc.Client: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] 18/05/14 05:06:52 WARN security.UserGroupInformation: PriviledgedActionException as:cloudera (auth:KERBEROS) cause:java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] ls: Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]; Host Details : local host is: "quickstart.cloudera/10.0.2.15"; destination host is: "quickstart.cloudera":8020; 5:39 PM (45 minutes ago) to me [cloudera@quickstart Desktop]$ env KRB5_TRACE=/dev/stdout kinit hdfs [10379] 1526299382.926005: Getting initial credentials for hdfs@HADOOPSEC.COM [10379] 1526299382.926163: Sending request (197 bytes) to HADOOPSEC.COM [10379] 1526299382.926190: Resolving hostname quickstart.cloudera [10379] 1526299382.926344: Sending initial UDP request to dgram 10.0.2.15:88 [10379] 1526299382.931792: Received answer from dgram 10.0.2.15:88 [10379] 1526299382.931809: Response was not from master KDC [10379] 1526299382.931831: Processing preauth types: 19 [10379] 1526299382.931839: Selected etype info: etype aes256-cts, salt "(null)", params "" [10379] 1526299382.931841: Produced preauth for next request: (empty) [10379] 1526299382.931844: Salt derived from principal: HADOOPSEC.COMhdfs [10379] 1526299382.931846: Getting AS key, salt "HADOOPSEC.COMhdfs", params "" Password for hdfs@HADOOPSEC.COM: [10379] 1526299384.710120: AS key obtained from gak_fct: aes256-cts/96F0 [10379] 1526299384.710174: Decrypted AS reply; session key is: aes256-cts/ABCF [10379] 1526299384.710187: FAST negotiation: available [10379] 1526299384.710206: Initializing FILE:/tmp/krb5cc_501 with default princ hdfs@HADOOPSEC.COM [10379] 1526299384.710293: Removing hdfs@HADOOPSEC.COM -> krbtgt/HADOOPSEC.COM@HADOOPSEC.COM from FILE:/tmp/krb5cc_501 [10379] 1526299384.710298: Storing hdfs@HADOOPSEC.COM -> krbtgt/HADOOPSEC.COM@HADOOPSEC.COM in FILE:/tmp/krb5cc_501 [10379] 1526299384.710334: Storing config in FILE:/tmp/krb5cc_501 for krbtgt/HADOOPSEC.COM@HADOOPSEC.COM: fast_avail: yes [10379] 1526299384.710346: Removing hdfs@HADOOPSEC.COM -> krb5_ccache_conf_data/fast_avail/krbtgt\/HADOOPSEC.COM\@HADOOPSEC.COM@X-CACHECONF: from FILE:/tmp/krb5cc_501 [10379] 1526299384.710350: Storing hdfs@HADOOPSEC.COM -> krb5_ccache_conf_data/fast_avail/krbtgt\/HADOOPSEC.COM\@HADOOPSEC.COM@X-CACHECONF: in FILE:/tmp/krb5cc_501 [cloudera@quickstart Desktop]$ klist -e Ticket cache: FILE:/tmp/krb5cc_501 Default principal: hdfs@HADOOPSEC.COM Valid starting Expires Service principal 05/14/18 05:03:02 05/15/18 05:03:02 krbtgt/HADOOPSEC.COM@HADOOPSEC.COM renew until 05/21/18 05:03:02, Etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96 Any information is highly appreciatable , HDFS daemon are all up and runining expect NAMENODE which complains about encryption type  ... View more

Hello    I just started learning sqoop in pseudo mode , Now I am trying to install sqoop in multinode cluster for the testing purpose , I stuck in the middel please help me.   I have 4 nodes  - with YARN ,hdfs configuraton and all the dameons.  Now  i install mysql in one node , i am sure when it is not a uber job it will spread across the nodes having said that how will access the mysql database that is installed in master node.    Look into the few forums for making mysql accessible from the remote host. inspite of doing all the below  still it says user@192.165.123.2 does not have access to the host.    1 . addedd the bind address in /mysql/my.cnf file 2 . gave all the privieges to user@192.165.123.2 in mysql as root. 3. flushed privileges.  4.all the other host have jdbc driver connections in the sqoop client.   is there any blog that gives a step by step installation sqoop in multinode cluster.    host 1 has sqoop installed and mysql  host 2  - i am executing the below    sqoop list-tables \ --connect jdbc:mysql://MasterNode/movieData \ --username user1 --password pass   ... View more

Folks    Could be a silly question or not . I really dont is there any certification path . like before taking CCA 505 Update  , i should have finished CCA 500. Or I can go straigh away and register for CCA 505.   please guide me .   Thanks   ... View more

Hi I am newbie , please help me    Beeline version 0.12.0-cdh5.0.0   Below are the steps are been performed    1. /etc/hive/conf/hive-site.xml   <configuration> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://guna:3306/metastore</value> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>hiveuser</value> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>password</value> </property> <property> <name>datanucleus.autoCreateSchema</name> <value>false</value> </property> <property> <name>datanucleus.fixedDatastore</name> <value>true</value> </property> <property> <name>datanucleus.autoStartMechanism</name> <value>SchemaTable</value> </property> <property> <name>hive.metastore.uris</name> <value>thrift://192.168.123.1:9083</value> </property> <property> <name>hive.support.concurrency</name> <value>true</value> </property> <property> <name>hive.zookeeper.quorum</name> <value>guna,tigerZ,hory</value> </property> </configuration> 2. Created  the Hive warehouse directory in HDFS.   3. Made the MySQL connector jar file accessible to Hive    4. Initialized the Hive Metastore in MySQL database   5. sudo service hive-metastore start  checked with sudo jps - able to see Runjar    6. sudo service hive-server2 start   When I execute the below beeline in my guna teriminal I am getting the below error  please help me    beeline -u jdbc:hive2://guna:10000/default -n username scan complete in 3ms Connecting to jdbc:hive2://guna:10000/default Error: Invalid URL: jdbc:hive2://guna:10000/default (state=08S01,code=0) Beeline version 0.12.0-cdh5.0.0 by Apache Hive 0: jdbc:hive2://guna:10000/default> Thanks in advance  ... View more

        I am receiving the below error , I uninstalled the zookeeper and installed again freshly the same error pops  new to CDH , could  you please let me know the syntax for --force option , searched in the wiki could not find.    below is the error i get      sudo service zookeeper-server init --myid=2 ZooKeeper data directory already exists at /var/lib/zookeeper (or use --force to force re-initialization)     ... View more

Below are the steps are been performed    1. /etc/hive/conf/hive-site.xml     <configuration> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://guna:3306/metastore</value> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>hiveuser</value> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>password</value> </property> <property> <name>datanucleus.autoCreateSchema</name> <value>false</value> </property> <property> <name>datanucleus.fixedDatastore</name> <value>true</value> </property> <property> <name>datanucleus.autoStartMechanism</name> <value>SchemaTable</value> </property> <property> <name>hive.metastore.uris</name> <value>thrift://192.168.123.1:9083</value> </property> <property> <name>hive.support.concurrency</name> <value>true</value> </property> <property> <name>hive.zookeeper.quorum</name> <value>guna,tigerZ,hory</value> </property> </configuration> 2. Created  the Hive warehouse directory in HDFS.   3. Made the MySQL connector jar file accessible to Hive    4. Initialized the Hive Metastore in MySQL database   5. sudo service hive-metastore start  checked with sudo jps - able to see Runjar    6. sudo service hive-server2 start   When I execute the below beeline in my guna teriminal I am getting the below error  please help me    beeline -u jdbc:hive2://elephant:10000/default \ -n username beeline -u jdbc:hive2://elephant:10000/default -n username scan complete in 3ms Connecting to jdbc:hive2://guna:10000/default Error: Invalid URL: jdbc:hive2://guna:10000/default (state=08S01,code=0) Beeline version 0.12.0-cdh5.0.0 by Apache Hive 0: jdbc:hive2://guna:10000/default>       ... View more

Hello All  I am new to hadoop .  using Hadoop 2.3.0-cdh5.0.0    In the process of  adding and decommission of datanode in multi node cluster . Read about the steps in the book.  I really got stuck in the middle as to what permission(user:group) should be set for include and exclude file which should reside on the local system of the namenode running . Also it will be much helpful if anyone can give me a example as to how include and exclude file will look.  I assume we just need to add the Network address of the datanode cluster in each of the file. Correct me if I am wrong.   Thanks   ... View more

  Hello    I am newbie , in process of taking snapshot . so created directory in the linux filesystem under my username and when i try to execute the below command in terminal , all daemons s are up and running .    Hadoop 2.3.0-cdh5.0.0   sudo -u hdfs hdfs dfsadmin -allowSnapshot /home/training/Desktop/backup I get the below errors    allowSnapshot: Directory does not exist: /home/training/Desktop/backup at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.valueOf(INodeDirectory.java:55) at org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.setSnapshottable(SnapshotManager.java:105) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.allowSnapshot(FSNamesystem.java:6868) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.allowSnapshot(NameNodeRpcServer.java:1229) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.allowSnapshot(ClientNamenodeProtocolServerSideTranslatorPB.java:1006) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1986) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1982) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1980) i did the check directory    [training@elephant backup]$ pwd /home/training/Desktop/backup I have no clue as to how to solve the above issue .    thanks ... View more

Hello folks    I am newbie , I could not find the minimum requirements pertain to the cores , as how many cores is sufficient to setup 3 node cluster in VM planning to. Upgrade my machine to 12GB Ram and 3 core Intel .   Please let me know if the above is sufficient .  **** Just a home build machine not in production  thanks   ... View more