Hi,    has anyone get this running and can post an running example ?    thx marcel      ... View more

@Raja Sekhar Chintalapati, The below command should do hdfs dfs -ls -R / | awk '{ if ( $3 == "spark" && substr($0,0,1) != "d" ) { print $8 } }' | xargs hdfs dfs -rm In the above command "spark" is the user name. Replace it with your username. Also I considered path as '/' . If you want to delete files only in a certain directory , replace it with your directory. This will remove only the files owned by the user and not the directories. Thanks, Aditya ... View more

@Raja Sekhar Chintalapati Here is your question already answered in our HCC forum ... View more

Hi, Is there a workaround solution for the above or is it a behaviour of 2.6.3 and above? ... View more

@Raja Sekhar Chintalapati You can you just need to toggle the button see attached screenshot. Here you have 2 storage destination Solr and HDFS. To use solr the default option since HDP 2.5 you need to store the configuration in zookeeper at least 3 for production but one is enough for test purposes,so you setup before activating. Go to Ambari Ui-->Ranger-->Config-->Ranger Audit when you log on ranger after you can enable all the plugins from Go to Ambari Ui-->Ranger-->Config-->Ranger Plugin UI Cheers ... View more

What changes you had to do in /etc/hosts file on RegionServer/Master? ... View more

@pbarna i tested firefox and i did make the changes you specified. issue here is not about the error message, it is how can we make http authentication against ad credentials to access any UI. I was able to access UI if i login with local credentials, i want to get the same when i login with my ad credentials when i login to domain pc. ... View more

You can login to resource manager url at port 8088 get the application Id specific to the submitted job,then run the following command to see the log info yarn logs -applicationId <application_id> | more ... View more

followed below document and able to see the status of NN & RM. http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.0/bk_Ambari_Security_Guide/content/_set_up_kerberos_for_ambari_server.html ... View more

You also must set in Firefox about:config network.auth.use-sspi = false to enable Kerberos. But most likely it still won't work because Windows doesn't know that your Oozie server etc. belong to another realm. Therefore install MIT Kerberos client for Windows, details how to install here, then copy krb5.conf from your cluster to "C:\Program Files\MIT\Kerberos\krb5.ini". Then, unlike in that article, change krb5.ini and set your default realm to your AD realm, and in the domain_realm section list all cluster master node FQDN's and set their realm to your HDP realm. After that restart your PC, and try to access Oozie Web UI. In the Kerberos Ticket Manager you can see which principals have been contacted, and that your cluster masters are in the right domain. ... View more

@Allen Wood Sorry you are going against the rules by publishing links to exam dumps. Please try to desist this practice in future. ... View more

This should have been automatically created for you if you entered CHRSV@COM in the "Additional Realms" box on the Configure Identities in the Enable Kerberos Wizard. Assuming that you didn't do this, how was the krb5.conf file set up to acknowledge the trusted realm? ... View more

centrify express is best option available & good part is it is free ... View more

Hi @Raja Sekhar Chintalapati, The documentation you mention is not specific to a MIT KDC. It is also perfectly valid when using an active directory. You just need to set hadoop.http.authentication.kerberos.principal and hadoop.http.authentication.cookie.domain according to your specific settings. The parameters are used to allow SPNEGO authentication when accessing some UIs of your cluster. Note that, in some cases, configuration on your browser might be required to use the tickets of your workstation. Example: https://community.hortonworks.com/articles/28537/user-authentication-from-windows-workstation-to-hd.html https://community.hortonworks.com/articles/76873/configure-mac-and-firefox-to-access-hdphdf-spnego.html Hope this helps. ... View more

@ Raja Sekhar Chintalapati Of course if you setup a KDC the keytabs are valid but you need to grab a valid one to proceed! Get the list of keytabs List all valid keytabs $ ls /etc/security/keytabs List valid principals for this keytab $ klist -kt /etc/security/keytabs/hive.service.keytab Keytab name: FILE:/etc/security/keytabs/hive.service.keytab KVNO Timestamp Principal ---- ----------------- -------------------------------------------------------- 1 02/02/17 23:00:12 hive/Ambari-Host_name@YOUR_REALM.COM 1 02/02/17 23:00:12 hive/Ambari-Host_name@YOUR_REALM.COM Grab a valid ticket $ kinit -kt /etc/security/keytabs/hive.service.keytab hive/Ambari-Host_name@YOUR_REALM.COM Check validity $ klist Ticket cache: FILE:/tmp/krb5cc_504 Default principal: hive/Ambari-Host_name@YOUR_REALM.COM Valid starting Expires Service principal 02/10/17 01:32:45 02/11/17 01:32:45 krbtgt/YOUR_REALM.COM@YOUR_REALM.COM renew until 02/10/17 01:32:45 Grab a valid ticket $ kinit -kt /etc/security/keytabs/hive.service.keytab hive/Ambari-Host_name@YOUR_REALM.COM This should have been the correct connect string if you had a valid ticket beeline -u jdbc:hive2://hiveServer2_hostname:10000;principal=hive/Keytab@PRINCIPAL With the above you should successfully log on and execute your HQL ... View more

You don't need any princs/keytabs for Oozie database, the only principal you need for Oozie is oozie/oozie_server_fqdn@REALM and it will be created by the Kerberos wizard you select to use. ... View more

Remember If you have KAFKA : you need to change at config -> kafka brokers -> listeners back to PLAINTEXT://localhost:6667 (from PLAINTEXTSASL://localhost:6667) ... View more

Hello Having the same issue 😞 I find in the hive meta store log 2017-03-10 16:50:52,164 INFO [main]: zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:user.name=hive No idea where this is coming from though All tips appreciated! Peter ... View more

@SBandaru @Prabhu M question guys-- what is the best sequence to make env Kerborized Local KDC for SP & AD for UP with no SSL. As of now i got ENV up and running with no kerborisation. ... View more

Ambari should properly kerberize your cluster. Did you restart all the affected services after enabling kerberos? ... View more

@Raja Sekhar Chintalapati: Can you please tell me if you find solution to this problem ... View more

@Sergey Soldatov or @Raja Sekhar Chintalapati Do you know if this still is the case? Is there any plans for this in the future? Thanks! It looks like the jira mentioned here is resolved ... View more

@Kuldeep Kulkarni great stuff. I find myself getting this confused as well. ... View more

Symptom Not able to use reflect function using beeline, but the query works OK with Hvi CLI. Error Message: Error while compiling statement: FAILED: SemanticException UDF reflect is not allowed (state=42000,code=40000) Cause When running set hive.server2.builtin.udf.blacklist from beeline, it will return the following as blacklisted: jdbc:hive2://localhost:10000/default> set hive.server2.builtin.udf.blacklist; +------------------------------------------------------------------+--+ | set | +------------------------------------------------------------------+--+ | hive.server2.builtin.udf.blacklist=reflect,reflect2,java_method | +------------------------------------------------------------------+--+ Reflect UDF is blacklisted by default when running queries through HiveServer2 (beeline, ODBC, JDBC connections), as it was found to be a security risk. The code was modified so if the parameter hive.server2.builtin.udf.blacklist has not been configured or it is blank, its default value will be "reflect,reflect2,java_method". Resolution 1. Open the Ambari UI 2. Add the custom property in Ambari hive.server2.builtin.udf.blacklist under Hive / Configs / Advanced / Custom hive-site and give it any value, for example "empty_blacklist". 3. Restart services as requested by Ambari. 4. Connect again with beeline and verify that blacklist only includes the dummy value now. 0: jdbc:hive2://localhost:10000/default> set hive.server2.builtin.udf.blacklist; +-------------------------------------------+--+ | hive.server2.builtin.udf.blacklist=empty_blacklist | +-------------------------------------------+--+ 5. Reflect should work now without issues. ... View more

@Kuldeep Kulkarni I got the same error message. only difference is my env is kerborized & both my rm's are not in standby mode. [yarn@m1 root]$ yarn rmadmin -getServiceState rm1 standby [yarn@m1 root]$ yarn rmadmin -getServiceState rm2 active Ambari doesn't show the state of RM, but getting same exception as above. i tried to switch the roles and that didnot help. Any help is appreciated. ... View more

@Neeraj Sabharwal yea...we did follow with support and they said it is a know issue...i posted the comments below..alll we need to do is change the DB engine for all the tables which are MyISAM to InnoDB. thanks for your response though.. ... View more